Question

I have a simple fcgi server written in c++ – for now it’s single threaded. I saw the main loop for example like

accept_connections();
handle_data();`
handle_connections();

but what if I want to respond to each client simultaneously? Because if I understand this, server first accepts new connections and iterate over them and proccess each. But this can be not good because of for example a upload (while one client uploads, others have to wait.) or by DoS like slowloris. How can server do multiple clients at one time ? I saw one server it was single-threaded – and did it.

How to do it ?

Answer 1

I solved it earlier but i want to answer my question.

Solution I made:

• 1 acceptor thread – (accept() for new incoming connections in
  blocking behavior) and assigning connections to network threads with load balancing
• N network threads where N = (1 thread per 1000 clients) with nonblocking behavior:
  • initialize
  • while server is running
    • set timeout watcher (for disabling blocking in event loop)
    • run event loop
    • stop timeout watcher
    • add new sockets from acceptor thread
    • iterate over connections run Update(), try to flush output, check for close, etc. if Update() return -1, erase connection from list
  • end while
  • release and close all connections and free all allocated memory
  • return
• Worker Threads with posix semaphores waiting for jobs (thread pool as management)
• (Optional) Background Thread with low priority for scheduling and checking, deleting for example tokens, sessions, etc. Lots of sleeping.

I wrote custom fastcgi handler. I also used libev for network threads, buffered and direct output and automatic waking and disabling write event. Debugging with valgrind. Compiler clang.

Reactions to other answers:

I didnt want to use thread per connection because it has some cost in spawning and also if there are lots of threads, there is also a slowdown and higher memory usage.

And I didnt use boost asio because I dislike frameworks and asynchronous IO was not one that i was looking for.