Question

We need to poll a web page every 5 minutes and the web page is growing rather large. The web page is a directory listing and we need the last line (to get a file name). What is the best way to get just this last line?

(If this was a local file I could position back a little relative to the end of file and read).

Answer 1

HTTP 1.1 does support a set of headers to request only a particular range of bytes, including support for just the last n bytes of a file (using the ‘suffix’ format). See here. For instance,

Range: bytes=-1000 

for the last 1000 bytes. (Assuming the server supports the Range header, of course.)