Question

I am writing an API, and I got stuck at the point where I am mixing asyncronous and syncronous code depending on the incoming request, take a look at the examples below.

routes.js

module.exports = [
    {
        method: 'GET',
        path: '/',
        controller: 'main',
        action: 'main',
        description: 'lists the API functionality',
        access: 'auth'
    },
    {
        method: 'POST',
        path: '/users',
        controller: 'users',
        action: 'create',
        description: 'creates a new user',
        fields: {
            fullName: {
                format: {
                    min: 2,
                    max: 64,
                    minWords: 2,
                    disableDoubleSpaces: true
                },
                description: 'the full name of the new user',
                examples: 'Thomas Richards, Richard Jones, Michael J. Fox, Mike Vercoelen, John Johnson'
            },
            email: {
                format: {
                    min: 2,
                    max: 64,
                    maxWords: 1,
                    match: 'email'
                },
                description: 'the email address of the new user',
                examples: '[e-mail], [e-mail], [e-mail], [e-mail]'
            },
            password: {
                format: {
                    min: 2,
                    max: 64
                },
                description: 'the password of the new user',
                examples: '123abcdfg, 7373kEjQjd, #8klKDNfk'
            }
        }
    }
];

the routes.js file is basically a very essential part of the API, it validates incoming data, routes to the correct controller/action and defines if the method is public, or requires authentication (basic auth).

api-server.js

var http = require('http');
var url = require('url');
var os = require('os');
var dns = require('dns');

var apiServer = module.exports;
var routes = require('./routes.js'); // Routes file from above.

var req, res, controller, action, serverInfo, httpServer;

apiServer.start = function(){
  prepare(function(){
    httpServer = http.createServer(handleRequest).listen(3000);
  });
};

//
// We need to do this function, we need the local ip address of the 
// server. We use this local ip address in logs (mongoDb) so we can
// refer to the correct server.
//
function prepare(callback){
  var serverName = os.hostname();

  dns.lookup(serverName, function(error, address){
    if(error){
      throw error;
    }

    serverInfo = {
      name: serverName,
      address: address
    };

    callback();
  });
}

function getRoute(){
  // loops through routes array, and picks the correct one...
}

function getAuth(callback){
  // parses headers, async authentication (mongoDB).
}

function getRequestData(callback){
  // req.on('data') and req.on('end'), getting request data.
}

function parseRequestData(callback){
  // parse request data at this point.
}

function validateRequestData(callback){
  // loop through route fields (see routes.js) and validate this data with the ones
  // from the request.
}

function requireControllerAndCallAction(){
  // do actual job.
}

function handleRequest(request, response){
  req = request;
  res = response;

  req.route = getRoute(); // First step for a request, syncronous.

  if(req.route === false){
    // 404...
  }

  // If in the routing schema access was "auth", 
  // this route requires authentication, so do that...
  if(req.route.access === 'auth'){
    getAuth(function(error, user){
      if(error){ // 401 } else {
        req.user = user;
      }
    }
  }

  if(req.method === 'POST' || req.method === 'PUT'){

    // Async functions.
    getRequestData(function(){
      parseRequestData(function(){
        validateRequestData(function(){
          requireControllerAndCallAction();
        });
      });
    });
  } else {
    requireControllerAndCallAction();
  }
}

As you can see, some functions are asyncronous (getAuth, getRequestData) and some are syncronous (parseRequestData, validateRequestData).

Now here is the thing:

Request 1. comes in with method POST, url ‘/users’ and data:

• fullName = ‘Rick’
• email: ‘rick@’
• password: ‘a’

So we loop through the workflow of the API:

1. get the current route (controller: users, action: create) see the second array element in routes.js

2. get the request data and on callback:
   a. parse data
   b. validate data

Now lets IMAGINE, the validation of the data takes 5 seconds (which is retarded, but just for example) and during that validation, a new request comes in, the new request is not handled untill the previous one was finished right?

Answer 1

If they were synchronous and if they did take 5 seconds then yes, for that instance of the server, requests would be blocked. That’s why it’s critical that blocking calls (network, db, file system etc…) calls are async. The event loop has to keep looping or the whole server blocks.

http://blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop/

A key line from that write up is:

…however, everything runs in parallel except your code

That means expensive I/O should be async but your code can block. but, it’s typically expensive I/O that we’re worried about blocking on the server.

‘Your code’ is typically just handling callbacks from the long running I/O calls, updating your state and then firing off another call. But that’s the beauty of it – when you’re updating your state, it’s on the main event loop so there’s no need for multi-threaded access to ‘your’ state and code. No locking, no deadlocks etc… but all the benefits of async and parallel I/O which is the expensive part.

The other key point (which 5 seconds of non-IO work would fall into) is:

Other than I/O calls, Node.js expects that all requests return
quickly; e.g. CPU-intensive work should be split off to another
process with which you can interact as with events, or by using an
abstraction like WebWorkers.

Also, do expect ‘auth’ and ‘POST’ | ‘PUT’ to ever happen in the same request? If so, you may have an issue. getAuth looks to be async but then you immediately go to the req.method check. At that point, getAuth will still be working. If you want POST and PUT authenticated then that block of async methods may need to be wrapped with the getAuth async method.