Question

I understand that generally the number of worker threads should be equal to the number of CPUs on your server (in my case it is 8), unless the thread is doing some kind of I/O etc.

My webserver provides services that require an extensive search in my mysql database, some queries takes about 16seconds in worst case. So two questions here,
1. How do I decide on the number of threads that will be optimal?
2. How can I simulate thousands of user and test my server against thousands of requests?

Answer 1

That depends on your definition of “threads”.

If your threads are “blocking”… as in they only handle one client at a time — 8 threads would be a terrible choice. If your worker threads all do non-blocking I/O, then yes, matching the threads to the cpu count would be a good choice.

The other thing to consider is whether your DB calls are non-blocking. I am not sure what language you are doing your scripting in (or are you doing that in C++ too?), but the c version of Mysql for example does blocking queries.

If you are looking for massive performance, I would take a look at NGiNX and G-WAN — they are the leaders in this area.

http://nginx.org/

http://www.trustleap.com/

Also, for load testing: http://httpd.apache.org/docs/2.0/programs/ab.html or http://www.hpl.hp.com/research/linux/httperf/