Question

I’m using Node’s request module.
The response I get is “gziped” or otherwise encoded.

How can I
1. Build the request to not encode the response?
2. Decode the response?
The data is coming from http://api.stackexchange.com.

var myRequest = require('request');
var zlib = require('zlib');

var stackRequest = require('request'); 
var apikey = '<MyKey>';
var fromdate = '1359417601';
var tagged = 'node.js';
stackRequest(
    { method: 'GET'
        , uri: 'http://api.stackexchange.com/2.1/questions?key=' + apikey + 
          '&site=stackoverflow&fromdate=' + fromdate + '&order=desc&' + 
          'sort=activity&tagged=' + tagged + '&filter=default'
}, function(err, response, body) { 
    console.log(response.body); // How can I decode this?
}); 

Answer 1

The encoding has nothing to do with request. StackOverflow’s API returns GZip encoded data always, as explained in the API documentation. You need to use Node’s zlib module to unzip the contents. This is a simple example:

var zlib = require('zlib');

// Other code

, function(err, response, body) {
   zlip.gunzip(body, function(err, data){
     console.log(data);
  });
});

The main downside of this, which is bad, is that this forces the request module to process the entire response content into one Buffer as body. Instead, you should normally use Node’s Stream system to send the data from the request directly through the unzipping library, so that you use less memory. You’ll still need to join the parts together to parse the JSON, but it is still better.

var zlib = require('zlib');
var request = require('request');

var apikey = '<MyKey>';
var fromdate = '1359417601';
var tagged = 'node.js';
var compressedStream = request('http://api.stackexchange.com/2.1/questions?' +
    'key=' + apikey + '&site=stackoverflow&fromdate=' + fromdate +
    '&order=desc&sort=activity&tagged=' + tagged + '&filter=default');

var decompressedStream = compressedStream.pipe(zlib.createGunzip());

var chunks = [];
decompressedStream.on('data', function(chunk){
  chunks.push(chunk);
});
decompressedStream.on('end', function(){
    var body = Buffer.concat(chunks);

    var data = JSON.parse(body);

    // Do your thing
});