Question

I am trying to learn about streams in node.js!

server.js

var net = require("net");

var server = net.createServer(function(conn) {

  conn.write("welcome!");

  # echo the user input!
  conn.pipe(conn);
});

server.listen("1111", function() {
  console.log("port 1111 opened");
});

telnet test

The server currently echos the user’s input

$ telnet localhost 1111
welcome!
hello
hello

desired output

To demonstrate where/how I should process the stream on the server side, I would like to wrap the user’s input in {} before echoing it back

$ telnet localhost 1111
welcome!
hello
{hello}

Answer 1

This will basically accomplish the exact output you’ve requested:

var net = require('net');

var server = net.createServer(function(c) {
    c.setEncoding('utf8');
    c.on('data', function(d) {
        c.write('{' + d.trim() + '}\n');
    });
});

server.listen(9871);

First let me call your attention to c.setEncoding('utf8'). This will set a flag on the connection that will automatically convert the incoming Buffer to a String in the utf8 space. This works well for your example, but just note that for improved performance between Sockets it would be better to perform Buffer manipulations.

Simulating the entirety of .pipe() will take a bit more code.

.pipe() is a method of the Stream prototype, which can be found in lib/stream.js. If you take a look at the file you’ll see quite a bit more code than what I’ve shown above. For demonstration, here’s an excerpt:

function ondata(chunk) {
  if (dest.writable) {
      if (false === dest.write(chunk) && source.pause) {
          source.pause();
      }
  }
}

source.on('data', ondata);

First a check is made if the destination is writable. If not, then there is no reason to attempt writing the data. Next the check if dest.write === false. From the documentation:

[.write] returns true if the entire data was flushed successfully to the kernel buffer. Returns false if all or part of the data was queued in user memory.

Since Streams live in kernel space, outside of the v8 memory space, it is possible to crash your machine by filling up memory (instead of just crashing the node app). So checking if the message has drained is a safety prevention mechanism. If it hasn’t finished draining, then the source will be paused until the drain event is emitted. Here is the drain event:

function ondrain() {
    if (source.readable && source.resume) {
        source.resume();
    }
}

dest.on('drain', ondrain);

Now there is a lot more we could cover with how .pipe() handles errors, cleans up its own event emitters, etc. but I think we’ve covered the basics.

Note: When sending a large string, it is possible that it will be sent in multiple packets. For this reason it may be necessary to do something like the following:

var net = require('net');

var server = net.createServer(function(c) {
    var tmp = '';
    c.setEncoding('utf8');
    c.on('data', function(d) {
        if (d.charCodeAt(d.length - 1) !== 10) {
            tmp += d;
        } else {
            c.write('{' + tmp + d.trim() + '}\n');
            tmp = '';
        }
    });
});

server.listen(9871);

Here we use the assumption that the string is ended by the new line character (\n, or ascii character code 10). We check the end of the message to see if this is the case. If not, then we temporarily store the message from the connection until the new line character is received.

This may not be a problem for your application, but thought it would be worth noting.