Question

I’ve set up a node server to save logging information. It is anticipated to have a very very high volume of requests coming in. I was working with a mongo data store, but that performance of that too unreliable. Reading the data on the reporting side of things often took an abnormally long time. So I switch to a classic, mysql. There is obviously significant write throughput advantages if you prepare bulk inserts as opposed to do singleton inserts. So I found this guy:

https://gist.github.com/3078520

I love it’s simplicity. It’s on a handful of lines and is very very powerful. It has two main functions add() which will add a record to the queue and then call flush() once the queue hits a certain size. And then it has flush() which joins the queue into one long string and creates a long bulk insert statement that is then shipped to the mysql driver (I simplified the function from the supplied link:

this.flush = function () {
/*1*/    var sql = queryTemplate.replace('{values}', '\n('+ queue.join('),\n(') +')');
/*2*/    handle.query(sql, callback);
/*3*/    queue = [];
}

This is essentially a class function that is part of a single class instance owned by the request handler. So my question is, is this safe? The queue is a class variable. When flush() is called, it handles the queue. The only problem I could see would be if flush() is called, line 1 is executed to make the sql statement (just a string), then another add() is called that adds something to the queue, then the first process calls the query() function and then empties the queue. In that case, the record that was added by the second process would be lost because the first process wouldn’t have known about it when it created the sql statement and would wipe it out when it clears the queue. Is this possible?

A more generalized form of this question would be as follows. Once a function begins executing, can anything else externally influence variables that have a scope outside of it? Here’s a very simple example, is it ever possible to do the console.log in this example presuming different requests call these two functions:

var global_number;
function a(){
  global_number=2;
}
function b(){
  global_number=1;
  if(global_number==2){
    console.log("a() influenced global_number in the middle of b()'s execution")
  }
}

This is mostly a yes/no question, but I’m looking for a little more than that. A brief explanation or a link to a page where I can learn more would be great! Thanks!

Answer 1

Yes, if the execution of functions are async, then you may have a problem with the data owned by a global variable. Here is the test scenario;

var globalVariable = 10;

function a ()
{
    var timer = setInterval(function() {
        globalVariable--;
        console.log(globalVariable)
    }, 1000);
}

function b ()
{
    setTimeout(function() {
        globalVariable = 20;
    }, 3000);
}

a();
b();

This will out put :

9
8
7
19
18
17
16

If you ask to function execution interruption which is not async, no way! See the example.

var globalVariable = 1000000000;
function a () {
    for(var i=0; i < 1000000000; i++)
    {
        if(globalVariable == 20000) console.log('From a() : ' + globalVariable);
        globalVariable--;
        if(globalVariable == 0) break;
    }
}

function b () {
    setTimeout(function() {
        console.log('From b() : ' + globalVariable);
        globalVariable = 20000;
        console.log('From b() : ' + globalVariable);
    }, 1);
}

b();
a();

This will always output:

From a() : 20000
From b() : 0
From b() : 20000

So, your questions answer is NO, you can not interrupt a sync function execution.

However; in your code snippet example, it seems to ok to empty the queue since you have already consumed the data in the queue with queue.join at line 1. On the other hand, this approach suffers from rollback operation. When the sql returns an error you may not be able to repopulate the queue, for example.