Question

I am building a java server that needs to scale. One of the servlets will be serving images stored in Amazon S3.

Recently under load, I ran out of memory in my VM and it was after I added the code to serve the images so I’m pretty sure that streaming larger servlet responses is causing my troubles.

My question is : is there any best practice in how to code a java servlet to stream a large (>200k) response back to a browser when read from a database or other cloud storage?

I’ve considered writing the file to a local temp drive and then spawning another thread to handle the streaming so that the tomcat servlet thread can be re-used. This seems like it would be io heavy.

Any thoughts would be appreciated. Thanks.

Answer 1

When possible, you should not store the entire contents of a file to be served in memory. Instead, aquire an InputStream for the data, and copy the data to the Servlet OutputStream in pieces. For example:

ServletOutputStream out = response.getOutputStream(); InputStream in = [ code to get source input stream ]; String mimeType = [ code to get mimetype of data to be served ]; byte[] bytes = new byte[FILEBUFFERSIZE]; int bytesRead;  response.setContentType(mimeType);  while ((bytesRead = in.read(bytes)) != -1) {     out.write(bytes, 0, bytesRead); }  // do the following in a finally block: in.close(); out.close(); 

I do agree with toby, you should instead ‘point them to the S3 url.’

As for the OOM exception, are you sure it has to do with serving the image data? Let’s say your JVM has 256MB of ‘extra’ memory to use for serving image data. With Google’s help, ‘256MB / 200KB’ = 1310. For 2GB ‘extra’ memory (these days a very reasonable amount) over 10,000 simultaneous clients could be supported. Even so, 1300 simultaneous clients is a pretty large number. Is this the type of load you experienced? If not, you may need to look elsewhere for the cause of the OOM exception.

Edit – Regarding:

In this use case the images can contain sensitive data…

When I read through the S3 documentation a few weeks ago, I noticed that you can generate time-expiring keys that can be attached to S3 URLs. So, you would not have to open up the files on S3 to the public. My understanding of the technique is:

1. Initial HTML page has download links to your webapp
2. User clicks on a download link
3. Your webapp generates an S3 URL that includes a key that expires in, lets say, 5 minutes.
4. Send an HTTP redirect to the client with the URL from step 3.
5. The user downloads the file from S3. This works even if the download takes more than 5 minutes – once a download starts it can continue through completion.