Question

I’m currently working on a simple NodeJS client that connects to a PHP server using the net classes. In addition, the NodeJS client is working as a Socket.IO server that sends data received from the PHP server to the browsers connected with Socket.IO.

So far, everything is working fine. Yet if I connect with another client to Socket.IO, the PHP server has to send a notification to every connected client. Thus, it sends a JSON-encoded array to the NodeJS client which processes the JSON data (decoding and modifying it a bit).

Now the problem is that sometimes two separate messages sent by the PHP server are concatenated in NodeJS’ onData event handling function:

client.on("data", function(data) {

   var msgData = JSON.parse(data.toString("utf8"));
   [...]
}

The variable data now sometimes (not every time!) contains two JSON-strings, such as:

 { "todo":"message", [...] } { "todo":"message", [...] }

This of course results in an exception thrown by the JSON.parse function. I expected two calls of the onData-function with the variable data being:

{ "todo":"message", [...] }

On the PHP server side I have to iterate over an array containing all Socket.IO-connections that are currently served:

foreach($sockets as $id => $client) {

    $nodeJS->sendData($client, array("todo" => "message", [...]);
}

The $nodeJS->sendData-function json-encodes the array and sends it to the NodeJS client:

socket_write($nodeClient, json_encode($dataToSend));

The $nodeJS->sendData function is definitively called two times, as socket_write is.

I now have no idea whether PHP or NodeJS concatenates those two strings. What I want, is that NodeJS calls the onData-handler once for each time the $nodeJS->sendData function is called (e.g. sendData is called twice → the onData-event is fired twice).
I could of course add some flag at the end of each json-encoded string and later split them into an array in the onData function. However, I don’t like that solution much.

Is there an easier way to accomplish this?

Answer 1

It’s important to remember that when you’re reading from a socket, the data is going to come in arbitrary chunks and its entirely up to your code to split them up into units that are meaningful to process; there is absolutely no guarantee that each chunk will correspond to one meaningful unit.

yannisgu has given you the first part of the solution (terminate each unit with a newline, so your code can tell where it ends): now you need to implement the second part, which is to buffer your incoming data and split it into units.

At initialization, do something like

var buf = '';

and set client‘s encoding to utf8.

In your "data" handler:
[UPDATED: incorporated josh3736’s suggestions]

buf += data;
var idx;
while ((idx = buf.indexOf('\n')) >= 0) {
   // there's at least one complete unit buffered up
   var unit = buf.slice(0, idx);
   // extract it
   if (unit.slice(-1) == '\r') {
     // CRLF delimited
     unit = unit.slice(0, -1);
   }
   if (unit.length) {
     // ignore empty strings
     var msgData = JSON.parse(unit);
     [...]
     // process it
   }
   buf = buf.slice(idx +1);
   // discard it
}
// at this point, buf is either empty or contains an incomplete
// unit which will be processed as soon as the rest of it comes in