Question

I writing downloader app. I want to only use java Socket to request a file. So I write in my socket just as HTTP protocol rules. My app creates a connection and after reading headers, uses read() method of InputStream of my socket. Everything goes well. sometimes connection may lost. but I store which byte I was reading, so again it creates a new Socket with HTTP Ranged GET and continue its work. But when download is about to finish, I mean when less than 10 KB remains, All Connections will lost and it again(as scheduled) try to open new Socket and Continue its work. It completely reads headers of response but before reading any byte of body, read() method returns -1 and Again and Again it tries to Open a new Socket and read() remaining bytes, but the problem persists. The point is every time response headers can completely be read. and I see the Content-Length: field of response header is exactly the remaining bytes of file. I forgot to mention: my code has a problem because I check many file from many servers and result is the same. here is the Code:

// Some fields:
int state;
long start, current, end;

// in a thread:
while (state != FINISHED) {
    if (state == DOWNLOADING) {
        try {
            // fill a new socket with Ranged GET [current, end]
            Socket s = initConnection();
            InputStream in = s.getInputStream();
            int readNo = 0;
            FileOutputStream out = getTempFile();
            byte[] buffer = new byte[1024];
            // read response headers successfully and prints them, request range is OK. a sample of its print is at the end of page
            readHeaders(in);
            while (state == DOWNLOADING && (readNo = in.read(buffer)) != -1) {                      
                current += readNo;
                out.write(buffer, 0, readNo);
            }
            if (readNo == -1) {
                // at nearly end of download always print this and values never changes, where usually they have 3000 byte difference
                System.out.println("**************> (" + current + " - " + end + ")");
            }
            if (currentByte == endByte) {
                state = FINISHED;
                //mergeParts();
                // code never reaches here
                dlInfo.checkAllPartsFinished();
            }
            out.flush();
            out.close();
            s.close();
        } catch (Exception e) {
            e.printStackTrace();
            state = ERROR;
            error = e.getMessage();
            errorRetry++;
        }
    } else if (state == PAUSED) {
        // ...
    } else ...
    }
}

a sample of response header at the end of file where nothing changes:

HTTP/1.1 206 Partial Content
Date: Mon, 21 May 2012 14:34:27 GMT
Server: Apache
Last-Modified: Sat, 21 Apr 2012 02:16:20 GMT
ETag: "4006d32e-f691e0-4be26fda00500"
Accept-Ranges: bytes
Content-Length: 7859
Content-Range: bytes 2012041-2019899/16159200
Connection: close
Content-Type: application/octet-stream

**************> (2012041 - 2019899)

I don’t know what problem is but whatever it is, it occurs at nearly the end of stream.
I’m completely confused, after many hours spending time. I would be grateful for Any help!

thanx

Answer 1

Are you layering a Buffered Reader/Stream on top of the InputStream in your readHeaders() method? my guess is that you are doing this, and that this buffered stream is reading more of the InputStream than you expect (since it is buffered). these bytes are then lost when you return from the readHeaders() method.

UPDATE:

just saw your last comment. that is exactly your problem. the BufferedReader is consuming part of the body bytes.