Question

I have a Java applet that uses the HTTPURLConnection class to upload very large files to an IIS-7 web-server. The applet chunks the files into pieces, then POSTs the pieces using fixed-length streaming to a PHP script.

Occasionally, when uploading file chunks, the network connection between the client and the server mysteriously drops. When this happens, my call to the writeBytes() method throws an IOException which I catch. After catching this exception, I drop down into my finally block where I try to clean things up. Because insufficient data was written to the connection (remember, this is fixed-length streaming), the attempt to close the output stream also fails. As a result, the connection seemingly "sticks around" (i.e. the underlying socket remains open). This seems to be verified by looking at the source code to the close() method for the StreamingOutputStream class (note the comment stating that the socket cannot be closed).

Question:

Is there a graceful way I can shut down after catching an IOException while writing out to an HTTPURLConnection? Telling the HTTPURLConnection object to disconnect() doesn’t seem to be good enough.

Additional info:

Here’s the first exception I see when the network gets hosed, thrown by my call to the writeBytes() method of the HTTPURLConnection class:

java.io.IOException: Error writing request body to server
    at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(Unknown Source)
    at java.io.DataOutputStream.write(Unknown Source)
    at MultiPartPostThread.run(MultiPartPostThread.java:321)

I catch this exception, and then attempt to close the DataOutputStream object that I was using to write to the connection. When I do so, I get this exception:

java.io.IOException: insufficient data written
    at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.close(Unknown Source)
    at java.io.FilterOutputStream.close(Unknown Source)
    at MultiPartPostThread.run(MultiPartPostThread.java:370)

I can verify that the socket is staying open, by noting that it takes 10 minutes for the failure entry to show up in the IIS log. That delay (10 minutes) happens to be my connection timeout value. Example lines from the IIS log (trimmed for brevity):

2012-01-31 20:20:07 POST /upload_handler.php 200 0 0 356 26215490 3666
2012-01-31 20:20:10 POST /upload_handler.php 200 0 0 356 26215490 3853
2012-01-31 20:30:22 POST /upload_handler.php 500 0 64 0 15286099 611442

Note that in the first two lines of the log we’re transferring along nicely: 25 MB is sent each time, and a 200 status code is returned. The failure then shows up 10 minutes after the last successful transfer, with an unhelpful error 500 (and note that this was only a partial transfer; only ~15MB transferred). In actuality, this mysterious disconnect occurs about 1 minute into the whole procedure, so the timeout messages I’m seeing in my IIS trace logs are red herrings. I see nothing helpful in my PHP logs, the HTTPERR log, or in the system log on the server.

Answer 1

The more I look into this, the more I’m convinced that a graceful cleanup cannot take place. A disconnect() call doesn’t seem to be enough to close the underlying socket; you simply have to wait for it to time out.

I think I’ll try using the Apache HttpComponents HttpClient (as recommended at the bottom of this terrific SO wiki post), to see if its built-in error handling can take care of this bizarre case I’m seeing.