Home/ Questions/Q 282947
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-12T05:19:55+00:00

I’ve got an application, written in C++, that uses boost::asio. It listens for requests

  • 0

I’ve got an application, written in C++, that uses boost::asio. It listens for requests on a socket, and for each request does some CPU-bound work (e.g. no disk or network i/o), and then responds with a response.

This application will run on a multi-core system, so I plan to have (at least) 1 thread per core, to process requests in parallel.

Whats the best approach here? Things to think about:

  • I’ll need a fixed size thread pool (e.g. 1 thread per CPU)
  • If more requests arrive than I have threads then they’ll need to be queued (maybe in the o/s sockets layer?)

Currently the server is single threaded:

  • It waits for a client request
  • Once it receives a request, it performs the work, and writes the response back, then starts waiting for the next request

Update:

More specifically: what mechanism should I use to ensure that if the server is busy that incoming requests get queued up? What mechanism should I use to distribute incoming requests among the N threads (1 per core)?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I don’t see that there is much to consider that you haven’t already covered.

    If it is truly CPU-bound then adding threads beyond the number of cores doesn’t help you much except if you are going to have a lot of requests. In that case the listen queue may or may not meet your needs and it might be better to have some threads to accept the connections and queue them up yourself. Checkout the listen backlog values for your system and experiment a bit with the number of threads.

    UPDATE:

    listen() has a second parameter that is your requested OS/TCP queue depth. You can set it up to the OS limit. Beyond that you need to play with the system knobs. On my current system it is 128 so it is not huge but not trivial either. Check your system and consider whether you realistically need something larger than the default.

    Beyond that there are several directions you can go. Consider KISS – no complexity before it is actually needed. Start off with something simple like just have a thread to accept connection (up to some limit) and plop them in a queue. Worker threads pick them up, process, write result, and close socket.

    At the current pace of my distro’s Boost updates (and my lack of will to compile it myself) it will be 2012 before I play with ASIO – so I can’t help with that.

    • 0