Question

I’m trying to build a quick and simple image uploading service with Node, that takes the received images and saves them to Mongo’s GridFS.

GridFS get requires a Buffer object NodeJS Mongo Driver GridFS put

The question is pretty simple: how do I exactly cast/transform the received request body into a proper buffer.

My code so far (only the important pieces):
api.js

var express = require('express');
var connect = require('connect');

var app = module.exports = express.createServer();

app.configure(function(){
  app.use(express.bodyParser()); 
  app.use(express.methodOverride());
  app.use(app.router);
});

var upload = require('./upload.js');
app.post('/upload', upload.upload);

upload.js

exports.upload = (function(req, res, next){
  console.log("Uploading image...");
  // Create buffer
  // Rest of the code
}

I’ve tried:

  var buffer = new Buffer(util.inspect(req.body),'binary');

Creates the buffer, but it has a wrong size and probably not the correct content since util.inspect is obviously not the right way to go.

And:
var buffer = new Buffer(req.body);
Result:

 [Decode error - output not utf-8][Decode error - output not utf-8]

Buffer length = 0

I’m quite new to both Node and JavaScript developing in general, so probably I’m missing something quite simple, don’t hesitate to point the obvious 🙂

Thanks!

Answer 1

First, remember that Express is built on top of Connect, which is the library that handles a large amount of the lower-level HTTP work, and it’s where bodyParser() comes from.

The body parser middleware internally uses Formidable to parse file uploads.

Formidable’s default behavior is to write uploaded files directly to disk – in other words, you don’t actually have access to the uploaded file stream within your route handler. You get the values of any regular form fields (<input>s) sent along in req.body, and you get uploaded file paths via req.files, but you don’t get file content.

The easy answer here is to simply read the file from disk and use that to save into Mongo, remembering to delete the temporary file when done. Of course, this introduces the unnecessary intermediate step of writing the file upload to a temporary folder and then loading to Mongo.

If you want to stream file data directly into Mongo, you have more of a challenge in front of you. You’ll have to write your own middleware to parse the upload stream.

This is actually relatively easy. You can just start with the internal Connect body parser implementation—using Formidable to do the heavy lifting—and use the onPart API to pass the stream back to your route handler so that you can pass it off to the Mongo driver.