Question

In Google App Engine, I tried reading a .txt file from a URL. Because the maximum allowed size is 1MB and the file is slightly larger, I’m using an alternative method described here.

So, what I’m trying to do is this:

FetchOptions fo = FetchOptions.Builder.allowTruncate().doNotFollowRedirects();
HTTPRequest request = new HTTPRequest(url,HTTPMethod.GET,FetchOptions.Builder.allowTruncate());
URLFetchService service = URLFetchServiceFactory.getURLFetchService();
HTTPResponse response = service.fetch(request);

My question is now, how can I read this response line by line? I’m trying to process each line which should be possible somehow as the source file is a simple text file.

I can get a byte[] with

byte[] content = response.getContent();

but I’m struggling with the further processing of it.

Or, can I do something completely different to achieve the same thing ?

I’m trying to read it line by line because I don’t need all the lines. Processing would be much easier than to put everything in one large string.

Answer 1

You can try:

ByteArrayInputStream bais = new ByteArrayInputStream(content);
BufferedReader reader = new BufferedReader(new InputStreamReader(bais, "utf-8"));

String line = null;
while ((line = reader.readLine()) != null) {
 ...
}

Alternatively, you can use IOUtils and call IOUtils.lineIterator(reader) (where reader is the InputStreamReader)