Question

I am playing with RserveCLI project, which is a .net client communicating with the statistical environment R. The basic idea is sending data/commends between this .NET client and an R session via TCP protocol.

One bug that others and I found is that big data trunk, say over 10k bytes, cannot get transfer successfully. I found the but in the following code snippet:

        // send the commend to R, then R will do some computation and get the data ready to send back
        int toConsume = this.SubmitCommand(cmd, data); 
        var res = new List<object>();
        while (toConsume > 0)
        {
            var dhbuf = new byte[4];
            if (this.socket.Receive(dhbuf) != 4)
            {
                throw new WebException("Didn't receive a header.");
            }

            byte typ = dhbuf[0];

            // ReSharper disable RedundantCast
            int dlength = dhbuf[1] + (((int)dhbuf[2]) << 8) + (((int)dhbuf[3]) << 16);

            // ReSharper restore RedundantCast
            var dvbuf = new byte[dlength];

            // BUG: I added this sleep line, without this line, bug occures
            System.Threading.Thread.Sleep(500);

            // this line cannot receive the whole data at once
            var received = this.socket.Receive(dvbuf);
            // so the exception throws 
            if (received != dvbuf.Length)
            {
                var tempR = this.socket.Receive(dvbuf);
                throw new WebException("Expected " + dvbuf.Length + " bytes of data, but received " + received + ".");
            }

The reason is that the .NET code runs too fast and the R side cannot send the data that fast. So the receive line after my inserted Thread.Sleep(500) does not get all the data. If I wait some time there, then it can get all the data. But I don’t know how long.

I have some basic idea to deal with the problem, for example, continuously use this.socket.Receive() to get data, but if there is no data there .Receive will block there.

I have little experience in socket programming, so I am asking the best practice for this kind of problem. Thanks!

Answer 1

According to the docs:

If you are using a connection-oriented Socket, the Receive method will read as much data as is available, up to the size of the buffer.

So you are never guranteed to get all the data asked for in the receive call. You need to check how many bytes were actually returned by the Receive, then issue another receive call for the remaining bytes. Continue that loop until you get all the bytes you were looking for.