Question

I need to be able to read a file, break it into packets of some arbitrary size, lets say 512 bytes, and send those packets over TCP. Problem is, the receiver isn’t getting all the bytes that im sending. If I send 1000 packets, the receiver blocks when reading from InputStream because he has no more data to read around 990 packets or so.

Here’s the code (just the sending and receiving parts):

Sender:

int parts = (int)Math.ceil((double)(file.length()/512.0));

out.println(parts+"");
int readFile;
int i = 0;
while ((readFile = fileIn.read(buffer)) != -1) {
   i++;
   fileOut.write(buffer, 0, readFile);
   fileOut.flush();
   System.out.println("-- Sent packet " + i + "/" + parts + ". " + "Bytes sent = " + readFile);
}

Receiver:

int parts = Integer.parseInt(in.readLine());
byte[] buffer = new byte[512];
FileOutputStream pw = new FileOutputStream("file.ext");
DataInputStream fileIn = new DataInputStream(socket.getInputStream());
for(int j = 0; j < parts; j++){
   int read = 0;
   if(j == parts - 1){
      read = fileIn.read(buffer);
      pw.write(buffer, 0, read);
   }else{
      fileIn.readFully(buffer);
      pw.write(buffer);
   }
   System.out.println("-- Received packet " + (j+1) + "/" + parts + ". Read " +read+ " bytes.");
}

I tried increasing the socket’s send and receive buffer size but without success. What am I missing?

Here is an output sample:

Sender:

-- Sent packet 1/10. Bytes sent = 512
-- Sent packet 2/10. Bytes sent = 512
-- Sent packet 3/10. Bytes sent = 512
-- Sent packet 4/10. Bytes sent = 512
-- Sent packet 5/10. Bytes sent = 512
-- Sent packet 6/10. Bytes sent = 512
-- Sent packet 7/10. Bytes sent = 512
-- Sent packet 8/10. Bytes sent = 512
-- Sent packet 9/10. Bytes sent = 512
-- Sent packet 10/10. Bytes sent = 234

Receiver:

-- Received packet 1/10. Read 512 bytes.
-- Received packet 2/10. Read 512 bytes.
-- Received packet 3/10. Read 512 bytes.
-- Received packet 4/10. Read 512 bytes.
-- Received packet 5/10. Read 512 bytes.
-- Received packet 6/10. Read 512 bytes.
-- Received packet 7/10. Read 512 bytes. (And it blocks here, because there is no more data to read)

Answer 1

TCP is a stream protocol. There are no packets, just a single stream of data.

You shouldn’t be making the assumption that a single write() (with or without a flush()) will correspond to a single read(). Therefore your receive loop for(int j = 0; j < parts; j++) is misguided: a better approach would be to count the number of bytes read an be prepared for read() calls returning varying amounts of data.

In the comments you argue that readFully() takes care of the problem. However, my concern lies not so much with the code as with the packet-based view of the stream. This leads to bugs like that in your final fileIn.read(buffer) call. It might return half the data that got sent as part of your last “packet”, and you’ll never know!