Question

Hi I am writing a simple nodejs push notification server which is basically reading a dynamically edited json file and pushing its content to the client.

UPDATE: Its working if I remove the like data = JSON.parse(data); can any one clarify please ?

I am editing the json file using python every 5 seconds. But every time I run the python loop to edit the file the Node Server crashes. However if I run the same code inside the while loop separately the Node Server works just fine.

I am getting the following error by the node server

   debug - cleared heartbeat timeout for client c_u5Oi1eGnwaDerGREMD
   debug - set heartbeat interval for client c_u5Oi1eGnwaDerGREMD
{"sample": {"name": "Shubhanshu Mishra100", "networks": ["facebook", "twitter",
"linkedin"]}}
   debug - websocket writing 5:::{"name":"notification","args":[{"sample":{"name
":"Shubhanshu Mishra100","networks":["facebook","twitter","linkedin"]},"time":"2
012-10-09T11:24:55.588Z"}]}
{"sample": {"name": "Shubhanshu Mishra100", "networks": ["facebook", "twitter",
"linkedin"]}}
   debug - websocket writing 5:::{"name":"notification","args":[{"sample":{"name
":"Shubhanshu Mishra100","networks":["facebook","twitter","linkedin"]},"time":"2
012-10-09T11:24:55.589Z"}]}
{"sample": {"name": "Shubhanshu Mishra200", "networks": ["facebook", "twitter",
"linkedin"]}}
   debug - websocket writing 5:::{"name":"notification","args":[{"sample":{"name
":"Shubhanshu Mishra200","networks":["facebook","twitter","linkedin"]},"time":"2
012-10-09T11:25:00.598Z"}]}
{"sample": {"name": "Shubhanshu Mishra200", "networks": ["facebook", "twitter",
"linkedin"]}}
   debug - websocket writing 5:::{"name":"notification","args":[{"sample":{"name
":"Shubhanshu Mishra200","networks":["facebook","twitter","linkedin"]},"time":"2
012-10-09T11:25:00.619Z"}]}


undefined:0

^
SyntaxError: Unexpected end of input
    at Object.parse (native)
    at F:\My Codes\NodeJs\PushNotification\server.js:25:16
    at fs.readFile (fs.js:176:14)
    at Object.oncomplete (fs.js:297:15)

I am reading the response.json file using the following code in Node.js

io.sockets.on('connection', function(socket){
    fs.watch('response.json', function(curr, prev){
        fs.readFile('response.json', 'utf8', function(err, data){
            if(err) throw err;
            console.log(data);
            data = JSON.parse(data);
            data.time = new Date();

            socket.volatile.emit('notification', data);
        });

    });
});

UPDATE: Its working if I remove the like data = JSON.parse(data); can any one clarify please ?

My python code which works when I invoke my python script each individually is:

import json
import time

jsonStr = {
    "sample": {
        "name": "Shubhanshu Mishra",
        "networks": [
                "facebook",
                "twitter",
                "linkedin"
        ]
    }
}

i = 0
jsonStr['sample']['name'] = "Shubhanshu Mishra" + str(i);
fStr = json.dumps(jsonStr)
with open('response.json', 'w') as f:
        f.write(fStr)
f.closed

However when I wrap the same code above in a while loop with a sleep of 5 seconds I get the above mentioned error. My code for the while loop is:

while True:
    i += 100
    print i
    jsonStr['sample']['name'] = "Shubhanshu Mishra" + str(i);
    fStr = json.dumps(jsonStr)
    #f = open("response.json", "w")
    with open('response.json', 'w') as f:
        f.write(fStr)
    f.closed
    #f.write(fStr)
    print "Written to file: " + fStr
    #f.close()
    time.sleep(5)

My full code can be seen at: https://github.com/napsternxg/PushNotification

Answer 1

What happens is this: Python writes the file by first clearing it, and then writing the string. Node fires the event as soon as the file is cleared, and reads the (empty) file. Then, trying to JSON parse an empty string, you get the error.

The question is, why doesn’t this happen when you run the simple python script (without the while loop) manually. I’m not sure, but I recon this has to do with all kinds of race conditions, the major difference being that in the loop, right after the write to the file, it prints some stuff, and sleeps some, where in the other case it just quits. I wouldn’t be surprised at all if you got the same (crashing) behaviour if you add a print and sleep to the manual while-free script as well.

So, how to solve this. A not so nice, but working approach would be to add a small sleep, to allow the write to finish:

io.sockets.on('connection', function(socket){
    fs.watch('response.json', function(curr, prev){
      setTimeout(function () {
        fs.readFile('response.json', 'utf8', function(err, data){
            if(err) throw err;
            console.log(data);
            data = JSON.parse(data);
            data.time = new Date();

            socket.volatile.emit('notification', data);
        });
      }, 50);
    });
});

Since I would expect you get 2 callbacks from the fs.watch, nicer would be to debounce them via underscore (in effect ignoring the first one):

var _ = require("underscore");
io.sockets.on('connection', function(socket){
    fs.watch('response.json', _.debounce(function(curr, prev){
        fs.readFile('response.json', 'utf8', function(err, data){
            if(err) throw err;
            console.log(data);
            data = JSON.parse(data);
            data.time = new Date();

            socket.volatile.emit('notification', data);
        }), 50);
    });
});

I think it would even work like you expect if you would ignore the error (because on the second callback from fs.watch, the file will contain the right data).

However, in the end, I would argue that this is not the way to shoot new data into a running program. I would use one of the following ways:

• Use a message queue system
• Use named pipes
• reload configurationfiles on sighup (and not when they change)