Question

Relating to the selected answer for Number of Concurrent Threads in CreateIoCompletionPort, I have a question regarding the definition of “runnable thread” in the documentation for I/O completion ports:

When the total number of runnable threads associated with the
completion port reaches the concurrency value, the system blocks the
execution of any subsequent threads associated with that completion
port until the number of runnable threads drops below the concurrency
value.

I understand that a thread, woken from GetQueuedCompletionStatus() because a completion packet was posted, might find itself in a sleep state for OTHER reasons (i.e., it has not yet completed its job and has not returned to call GetQueuedCompletionStatus(), but is sleeping and therefore – if I understand correctly – is not runnable according to the above definition).

What I would specifically like to know is this: Suppose the above thread, in its processing, is sleeping because it has spawned another thread and is waiting on that other thread to complete?

Will the thread be considered to be in a ‘runnable’ state if its child thread is in a runnable state – or not?

If not, it seems that one cannot spawn a worker thread from a thread associated with a completion port without risking the danger that other threads will wake up, such that the total number of running threads exceeds NumberOfConcurrentThreads.

For example – suppose I have 50 threads waiting on GetQueuedCompletionStatus(), the number of concurrent threads is 5, 20 completion packets are simultaneously posted that trigger very long tasks, and every time a thread wakes to perform one of these tasks, it immediately spawns a single new thread to do the work and waits on that thread to complete the long task before returning back to call GetQueuedCompletionStatus() once again.

In this example, would 20 threads wake up and spawn 20 total child threads that work in parallel, or would it only be 5 threads?

(Note: ADDENDUM: I am implementing a pool of worker threads using Boost.Asio in my specific case – which utilizes I/O completion ports internally – and my worker threads use JNI to call into Java; internally, the Java code spawns its own worker threads – so this is what concerns me.)

Answer 1

The place to look for answers to questions like this is always “Windows Via C/C++ (PRO-Developer)” by Jeffrey Richter and Christophe Nasarre.

On page 327 of my copy it clearly explains that the I/O completion port tracks the thread id of threads that it releases from a call to GetQueuedCompletionStatus() (or equivalent) and counts those threads as “running” as long as they don’t call any function that puts them in a wait state.

So, in your example, assuming the thread that calls GetQueuedCompletionStatus() is waiting on the thread that is created by JNI then the thread is considered as “not running” and so the IOCP can release another thread from GetQueuedCompletionStatus() to service requests. If all of your threads will eventually call into JNI and create another thread then I think you have a problem with your design as you are basically creating a “thread per event” design which will likely fail to scale as well as an IOCP design which doesn’t create external threads…

Is it a function of the JNI layer that spawns the thread or is it purely in the Java code you’re calling into? If the later then I suggest redesigning the java code if possible…