Question

I’m working on an application that gets information from a fixed web page and converts the information to a graph. The webpage I’m trying to receive is a txt file. The problem is that the txt file is so large that it takes more than a minute to get the file into a String in java. The code I’m using for this is as follows:

HttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet("http://winterberg.dnsalias.net:9000/log.txt");
ResponseHandler<String> handler = new BasicResponseHandler();
response = client.execute(httpGet, handler);

The txt file represents the history of temperatures in a house. Every hour one line of text is automatically added to the textfile, since 11 Aug 2011, so it contains more than 6000 lines. However, only the information of the last month is relevant for the application. Is there a way to start reading from the end of the file until it has read a certain number of lines?

Or is there another way to speed up reading the file?

I am not able to change the textfile.

Answer 1

You can read your text file fully and save the reading number of bytes. So, in the next call, you can read the range of bytes that start from the number that you store it and after download this specific bytes, append it on your original text file. and you have the text file that updated with the new lines.

You can use the part of code as follow to get the range of bytes :

DefaultHttpClient httpClient = new DefaultHttpClient();
HttpGet get = new HttpGet(new URI(...));
get.addHeader("Range", "bytes=" + start + "-");
InputStream in = httpClient.execute(get).getEntity().getContent();

And for append the file, you can open the file with append mode, and write new bytes on it :

FileOutputStream fos = new FileOutputStream("file.txt", true); //true means openning file in append mode

byte[] buffer = new byte[1024];
int nread = -1;
while((nread != in.read(buffer, 0, 1024))
    fos.write(buffer, 0, nread);

in.close();
fos.close();

Hope it’s useful for you 🙂