Question

Is there a known algorithm for implementing a connection pool? If not what are the known algorithms and what are their trade-offs?
What design patterns are common when designing and programming a connection pool?
Are there any code examples implement a connection pool using boost.asio?
Is it a good idea to use a connection pool for persisting connections (not http)?
How is threading related to connection pooling? When do you need a new thread?

Answer 1

If you are looking for a pure thread-pooling policy (may be a connection or any resource) there are two simple approaches viz:-

1. Half Sync/Half Async Model (usually using using message queues to pass information).

2. Leaders/Followers Model (usually using request queues to pass information).

The first approach goes like this:-

• You create a pool of threads to
  handle a resource. Often this size
  (number of threads) needs to be
  configurable. Call these threads
  ‘Workers‘.
• You then create a master thread that
  will dispatch the work to the
  Worker threads. The application program dispatches the task as a
  message to the master thread.
• The master thread puts the same on
  the message Q of a chosen Worker
  thread and the Worker thread removes itself from the
  pool. Choosing and removing the
  Worker thread needs synchronization.
• After the Worker completes the
  task, it returns to the thread-pool.

The master thread itself can consume the tasks it gets in FCFS or a prioritized manner. This will depend on your implementation.

The second model (Leader/Followers) goes something like this:-

• Create a thread pool. Initially all
  are Workers. Then elect a
  Leader, automatically rest-all become followers. Note that electing
  a Leader has to be synchronized.
• Put all the data to be processed on a
  single request Q.
• The thread-pool Leader dequeues
  the task. It then immediately
  elects a new Leader and starts executing the task.
• The new Leader picks up the next
  task.

There may be other approaches as well, but the ones outlined above are simple that work with most use-cases.

Half Sync/Half Async Major Weakness:-

• Higher context switching,
  synchronization, and data copying
  overhead.

Leader/Follwers Major Weakness:-

• Implementation complexity of
  Leader election in thread pool.

Now you can decide for yourself the more correct approach.
HTH,