Question

I have written a code which streams bytes from another server to my server and then I write those contents into my local file. It works fine when I use the read() method which doesn’t buffer data. But when I use buffering (intention is that I believe streaming would be faster for big files) I use read(byte[]) method and it gets only partial data while streaming. I am posting the code. Can any one pls point out the mistake or concepts what I am missing.

The following code works fine. (no streaming)

    private void doViewDocument(HttpServletRequest request,
            HttpServletResponse response, DocumentServletService servletService) throws GEMException, MalformedURLException, ProtocolException, IOException {

        final String objectID = request.getParameter(PARAM_OBJECT_ID);

        LOGGER.info("For Viewing Document objectID received from Request == " + objectID);

        if (GEMSharedUtil.isValidObjectId(objectID)) {

            String ebesDocDownloadURL = servletService.getDocumentDownloadUrl(objectID);

            if (!GEMSharedUtil.isValidString(ebesDocDownloadURL)) {             
                //response.setStatus(HttpServletResponse.SC_NOT_ACCEPTABLE);
                response.setHeader("ResponseStatus", "Not_OK");
                throw new GEMException();
            } else {
                HttpURLConnection con = null;
                BufferedInputStream bin = null;
                BufferedOutputStream bout = null;

                try {
                    con = (HttpURLConnection) new URL(ebesDocDownloadURL).openConnection();

                    WASSecurity.preauthenticateWithLTPACookie(con);
                    con.setRequestMethod(REQUEST_METHOD_GET);
                    con.setDoOutput(true); // Triggers POST but since we have set request method so it will override it
                    con.setDoInput(true);
                    con.setUseCaches(false);
                    con.setRequestProperty("Connection", "Keep-Alive");
                    con.setAllowUserInteraction(false);
                //  con.setRequestProperty("Content-Type",
                //  "application/octet-stream");

                    response.setBufferSize(1024);
                    response.setContentType(con.getContentType());
                    response.setContentLength(con.getContentLength());
                    response.setHeader("ResponseStatus", "OK");
                    response.setHeader("Content-Disposition", con
                            .getHeaderField("Content-Disposition"));

                    bin = new BufferedInputStream((InputStream)
                            con.getInputStream(), 1024);

                    bout = new BufferedOutputStream(
                            response.getOutputStream(), 1024);

                    byte[] byteRead = new byte[1024];


                    File file = new File("C:\\Documents and Settings\\weakStudent\\Desktop\\streamed\\testStream.pdf");

                    FileOutputStream fos = new FileOutputStream(file);

                    if(file.length() > 0) {
                        file.delete();
                    }
                    file.createNewFile();

                    BufferedOutputStream fbout = new BufferedOutputStream((OutputStream) fos);
                    int c;
                    while((c= bin.read()) != -1) {
                        bout.write(c);
                        fbout.write(c);
                    }
fos.close();
                    bout.flush();
                    fbout.flush();
                    fbout.close();
                    LOGGER.info("con.getResponseCode():" + con.getResponseCode());

                } finally {
                    try {
                        if (bout != null) {
                            bout.close();
                        }
                    } catch (IOException e) {
                        LOGGER.log(Level.SEVERE, e.getMessage(), e);
                        throw new RuntimeException(e);
                    } finally {
                        try {
                            if (bin != null) {
                                bin.close();
                            }
                        } catch (IOException e) {
                            LOGGER.log(Level.SEVERE, e.getMessage(), e);
                            throw new RuntimeException(e);
                        } finally {
                            if (con != null) {
                                con.disconnect();
                            }
                        }
                    }
                }
            }

        }   //if ends

    }

Now if I have the following while loop it doesn’t work correctly.

                while(bin.read(byteRead) != -1) {
                    bout.write(byteRead);
                    fbout.write(byteRead);
                }

Q2)
Also would like to know is it mandatory to use BufferedInputStream/BufferedOutputStream for streaming. For eg if I use following code snippet it works

    BufferedInputStream bin = null;

    try {
        //in = request.getInputStream();        

        bin = new BufferedInputStream((InputStream) request
                .getInputStream(), 1024);

        int respcode = HttpURLConnection.HTTP_OK;
        con = createConnection(uploadURL, REQUEST_METHOD_POST);
        con.setRequestProperty("X-File-Name",fileName);

        conOut = con.getOutputStream();
        bout = new BufferedOutputStream(conOut);
        byte[] byteRead = new byte[1024];       

        while (bin.read(byteRead) != -1) {
            bout.write(byteRead);
        }
        bout.flush(); 
        respcode = con.getResponseCode();    

But the following again streams partially (no BufferedInputStream used here)

    ServletInputStream in = null;

    try {
        in = request.getInputStream();      

        int respcode = HttpURLConnection.HTTP_OK;
        con = createConnection(uploadURL, REQUEST_METHOD_POST);
        con.setRequestProperty("X-File-Name",fileName);

        conOut = con.getOutputStream();
        bout = new BufferedOutputStream(conOut);
        byte[] byteRead = new byte[1024];       

        while (in.read(byteRead) != -1) {
            bout.write(byteRead);
        }
        bout.flush(); 
        respcode = con.getResponseCode();    

Answer 1

A1. Your discarding the number of bytes that were read, telling the output stream to write the entire contents of byteRead buffer, which may contain data from the previous read

int bytesIn = -1;
while((bytesIn = bin.read(byteRead)) != -1) {
    bout.write(byteRead, 0, bytesIn);
    fbout.write(byteRead, 0, bytesIn);
}

UPDATE
Basically, all your examples are suffering from the same problem. Your buffer is n bytes long, but the read can return 0 to n bytes in the buffer, you need to take note of the number of bytes the read method returns in order to know how much to write