Question

I need to convert a huge file into the following format: for each byte, print that byte as character and print its decimal value in brackets, so that “@” becomes “@[64]”. I’ve got the following:

var s = fs.createReadStream(path,  { encoding: 'binary' });  
var p = [];

s.addListener('data', function(data) {     
    for(var i = 0; i < data.length; i++) {
            p.push(data.charAt(i)  + "[" + data.charCodeAt(i) + "]");
    }
});

s.addListener('end', function(data) {     
    console.log(p.join(""));
});

This kind of works, however, is totally slow. Any ideas how this can be optimized?

upd. as per the comment, added some date statements and found out that the “p.push” line takes most of the time. So I guess the problem is not in file reading. Still, the question remains – how to speed this up.

Answer 1

If the file is utf-8, you’re going to probably want to decode it first. Also, don’t use the binary encoding, it’s basically deprecated by now.

Here’s an example:

var fs = require('fs')
  , StringDecoder = require('string_decoder').StringDecoder;

var stream = fs.createReadStream('/path/to/my/file')
  , decode = new StringDecoder('utf8')
  , out = [];

stream.on('data', function(buff) {
  buff = decode.write(buff);
  for (var i = 0, l = buff.length; i < l; i++) {
    out.push(buff[i] + ' [' + buff.charCodeAt(i) + ']');
  }
});

stream.on('end', function() {
  console.log(out.join('\n'));
});

To make it simpler, you can also use the built-in decoding feature of read streams: stream.setEncoding('utf-8'); instead of making your own StringDecoder, but it can be useful using your own in some cases, e.g. you still have direct access to the buffer.

Edit: Sorry, didn’t see you were asking for an optimization. What you’re doing is inherently slow. I thought you only wanted it for debugging purposes. There might just be a better way of doing what you’re trying to do in the first place. However, since you’re buffering everything anyway, you might as well just do an fs.readFile and walk over the buffer.