Question

I have a (stupid) node.js server that is here for the sole purpose of calling a function on the data it’s passed with each request and answer the result of that function. Here is the code I use:

var pageDown = require('./PageDown/Markdown.Sanitizer').getSanitizingConverter(),
 http = require('http');

http.createServer(function (req, res) {
  var data = "";
  res.writeHead(200, {'Content-Type': 'text/plain'});
  req.on('data', function (chunk) {
    data += chunk;
  });
  req.on('end', function() {
    res.end(pageDown.makeHtml(data));
  });
}).listen(1337, '127.0.0.1');
console.log('HServer running at http://127.0.0.1:1337/');

I use this server from python with the following code (atm I’m just benchmarking so it’s just a stress test):

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import requests

for i in range(1, 100000):
    r = requests.post('http://localhost:1337/', data="lol")
    print i
print "I'm done :')"

My problem is that this way of getting things done is slow. I have a large database that needs to be processed with this javascript function and I’m looking for ways to make the above process quicker. So suggestions are welcome!

Answer 1

If all your are doing is calling the node process to cleanup your HTML, why not just do that in python? I assume the markdown converter you are pointing to is this one: http://code.google.com/p/pagedown/source/browse/Markdown.Sanitizer.js. If that is the case, why not convert it to python (or find similar python code), and work with it in process. It seems nuts to make a network call to simply run this process since it will be much much slower than just doing it all in python. Here are some python markdown tools I was able to find: http://packages.python.org/Markdown/, https://github.com/trentm/python-markdown2.