Question

I’m mucking about with node.js and have discovered two ways of reading a file and sending it down the wire, once I’ve established that it exists and have sent the proper MIME type with writeHead:

// read the entire file into memory and then spit it out

fs.readFile(filename, function(err, data){
  if (err) throw err;
  response.write(data, 'utf8');
  response.end();
});

// read and pass the file as a stream of chunks

fs.createReadStream(filename, {
  'flags': 'r',
  'encoding': 'binary',
  'mode': 0666,
  'bufferSize': 4 * 1024
}).addListener( "data", function(chunk) {
  response.write(chunk, 'binary');
}).addListener( "close",function() {
  response.end();
});

Am I correct in assuming that fs.createReadStream might provide a better user experience if the file in question was something large, like a video? It feels like it might be less block-ish; is this true? Are there other pros, cons, caveats, or gotchas I need to know?

Answer 1

A better approach, if you are just going to hook up “data” to “write()” and “close” to “end()”:

// 0.3.x style
fs.createReadStream(filename, {
  'bufferSize': 4 * 1024
}).pipe(response)

// 0.2.x style
sys.pump(fs.createReadStream(filename, {
  'bufferSize': 4 * 1024
}), response)

The read.pipe(write) or sys.pump(read, write) approach has the benefit of also adding flow control. So, if the write stream cannot accept data as quickly, it’ll tell the read stream to back off, so as to minimize the amount of data getting buffered in memory.

The flags:"r" and mode:0666 are implied by the fact that it is a FileReadStream. The binary encoding is deprecated — if an encoding is not specified, it’ll just work with the raw data buffers.

Also, you could add some other goodies that will make your file serving a whole lot slicker:

1. Sniff for req.headers.range and see if it matches a string like /bytes=([0-9]+)-([0-9]+)/. If so, you want to just stream from that start to end location. (Missing number means 0 or “the end”.)
2. Hash the inode and creation time from the stat() call into an ETag header. If you get a request header with “if-none-match” matching that header, send back a 304 Not Modified.
3. Check the if-modified-since header against the mtime date on the stat object. 304 if it wasn’t modified since the date provided.

Also, in general, if you can, send a Content-Length header. (You’re stat-ing the file, so you should have this.)