Question

Sorry for the vague question.. but I’m not sure quite what the problem is.
I have a node http server that I’m using to serve JSON data to a web app. It works great, but my JSON strings are starting to get large(10-12 MB), so I want to add compression with zlib.

The JSON data is in a string variable and I want to compress and then write to the response object… but the results that go back to the client seem to always have with perfect headers, and no content. Here is my deliverResponse function:

var deliverResult = function (data, response, callback, acceptEncoding){
    var payload = callback + '(' + JSON.stringify(data) + ');';

    if (acceptEncoding.match(/\bdeflate\b/)) {
        response.writeHead(200, { 'Content-Encoding': 'deflate', 'Content-Type': 'text/javascript; charset=UTF-8' });
        zlib.deflate(payload, function(err, result){
            if(!err){
                //console.log(result.toString('utf8')); // I have data on the console here
                response.write(result.toString('utf8')); // No data sent here
           }
        });
    } else if (acceptEncoding.match(/\bgzip\b/)) {
        response.writeHead(200, { 'Content-Encoding': 'gzip', 'Content-Type': 'text/javascript; charset=UTF-8' });
        zlib.gzip(payload, function(err, result){
           if(!err){
                response.write(result.toString('utf8'));
           }
        });
    } else {
        writelog('INFO', 'Returning data without compression\n');
        response.writeHead(200, { 'Content-Type': 'text/javascript; charset=UTF-8' });
        response.write(payload);
    }

    response.end();
}

The http server examples with zlib use streams and the pipe function, but I’m not sending a file as I generate the JSON data in the app from a database, so I am basing this on the convenience method examples. My troubleshooting so far I know that the response object is good, and that result.toString(‘utf8’) outputs gobeldy-gook as expected. If I don’t send an acccept-encoding header to the server, it sends plain text perfectly – so it had got to be the compression functions.

Anyone have any idea about this? I’m pretty sure it has to to with my lack of understanding about streams, pipes, buffers and the zlib object, and it’s probably just a syntax issue, so hopefully someone who understands all this can help me out 🙂

Cheers

Answer 1

Solved….

Stupid problem.. the response.write is being called in a async function, so it’s doing the write.end() before the response.write and an empty response is sent… replaced the response.write with response.end in the callback and it works perfectly 🙂