Question

I’m combining the async and request modules to make api requests asynchronously and with rate limiting.

Here is my code

var requestApi = function(data){

    request(data.url, function (error, response, body) {

        console.log(body);

    });
};


async.forEachLimit(data, 5, requestApi, function(err){
      // do some error handling.
});

Data contains all the urls I make request to. Am limiting the number of concurrent request to 5 using forEachLimit method. This code makes the first 5 request then stops.

In the async docs it says "The iterator is passed a callback which must be called once it has completed". But I don’t understand this, what should I be doing to signal that the request has completed?

Answer 1

First, you shall add callback to your iterator function:

var requestApi = function(data, next){
    request(data.url, function (error, response, body) {
        console.log(body);
        next(error);
    });
};

next(); or next(null); tells Async that all processing is done. next(error); indicates an error (if error not null).

After processing all requests Async calls its callback function with err == null:

async.forEachLimit(data, 5, requestApi, function(err){
    // err contains the first error or null
    if (err) throw err;
    console.log('All requests processed!');
});

Async calls its callback immediately after receiving the first error or after all requests completed succesfully.