Question

I have a Node.js server which dynamically generates and serves small (200×200) thumbnails from images (640×640) in a database (mongodb). I’m using the node-imagemagick module for thumbnailing.

My code works roughly 95% of the time; about 1 in 20 (or fewer) thumbnailed images are corrupt on the client (iOS), which reports:

JPEG Corrupt JPEG data: premature end of data segment

For the corrupt images, the client displays the top 50% – 75% of the image, and the remainder is truncated.

The behavior is non-deterministic and the specific images which are corrupt changes on a per-request basis.

I’m using the following code to resize the image and output the thumbnail:

im.resize({
  srcData: image.imageData.buffer,
  width: opt_width,
}, function(err, stdout) {
  var responseHeaders = {};
  responseHeaders['content-type'] = 'image/jpeg';
  responseHeaders['content-length'] = stdout.length;
  debug('Writing ', stdout.length, ' bytes.');
  response.writeHead(200, responseHeaders);
  response.write(stdout, 'binary');
  response.end();
});

What could be wrong, here?

Notes:

1. The problem is not an incorrect content-length header. When I omit the header, the result is the same.
2. When I do not resize the image, the full-size image always seems to be fine.
3. In researching this I found this and this StackOverflow questions, which both solved the problem by increasing the buffer size. In my case the images are very small, so this seems unlikely to be responsible.
4. I was originally assigning stdout to a new Buffer(stdout, 'binary') and writing that. Removing it ('binary' will be deprecated) made no difference.

Answer 1

The problem seems to have been due to a slightly older version of node-imagemagick (0.1.2); upgrading to 0.1.3 was the solution.

In case this is helpful to anyone, here’s the code I used to make Node.js queue up and handle client requests one at a time.

// Set up your server like normal.
http.createServer(handleRequest);
// ...

var requestQueue = [];
var isHandlingRequest = false;  // Prevent new requests from being handled.

// If you have any endpoints that don't always call response.end(), add them here.
var urlsToHandleConcurrently = {
  '/someCometStyleThingy': true
};

function handleRequest(req, res) {
  if (req.url in urlsToHandleConcurrently) {
    handleQueuedRequest(req, res);
    return;
  }
  requestQueue.push([req, res]);  // Enqueue new requests.
  processRequestQueue();          // Check if a request in the queue can be handled.
}

function processRequestQueue() {
  // Continue if no requests are being processed and the queue is not empty.
  if (isHandlingRequest) return;
  if (requestQueue.length == 0) return;

  var op = requestQueue.shift();

  var req = op[0], res = op[1];

  // Wrap .end() on the http.ServerRequest instance to
  // unblock and process the next queued item.
  res.oldEnd = res.end;
  res.end = function(data) {
    res.oldEnd(data);
    isHandlingRequest = false;
    processRequestQueue();
  };

  // Start handling the request, while blocking the queue until res.end() is called.
  isHandlingRequest = true;
  handleQueuedRequest(req, res);
}

function handleQueuedRequest(req, res) {
  // Your regular request handling code here...
}