Question

What is the difference in using the blocking call pop() as compared to,

while(pop_if_present(...)) 

Which should be preferred over the other? And why?

I am looking for a deeper understanding of the tradeoff between polling yourself as in the case of while(pop_if_present(...)) with respect to letting the system doing it for you. This is quite a general theme. For example, with boost::asio I could do a myIO.run() which blocks or do the following:

while(1) 
{
myIO.poll()
}

One possible explanation is is that the thread that invokes while(pop_if_present(...)) will remain busy so this is bad. But someone or something has to poll for the async event. Why and how can this be cheaper when it is delegated to the OS or the library? Is it because the OS or the library smart about polling for example do an exponential backoff?

Answer 1

Intel’s TBB library is open source, so I took a look…

It looks like pop_if_present() essentially checks if the queue is empty and returns immediately if it is. If not, it attempts to get the element on the top of the queue (which might fail, since another thread may have come along and taken it). If it misses, it performs an “atomic_backoff” pause before checking again. The atomic_backoff will simply spin the first few times it’s called (doubling its spin loop count each time), but after a certain number of pauses it’ll just yield to the OS scheduler instead of spinning on the assumption that since it’s been waiting a while, it might as well do it nicely.

For the plain pop() function, if there isn’t anything in the queue will perform atomic_backoff waits until there is something in the queue that it gets.

Note that there are at least 2 interesting things (to me anyway) about this:

• the pop() function performs spin waits (up to a point) for something to show up in the queue; it’s not going to yield to the OS unless it has to wait for more than a little short moment. So as you might expect, there’s not much reason to spin yourself calling pop_if_present() unless you have something else you’re going to do between calls to pop_if_present()

• when pop() does yield to the OS, it does so by simply giving up it’s time slice. It doesn’t block the thread on a synchronization object that can be signaled when an item is placed on the queue – it seems to go into a sleep/poll cycle to check the queue for something to pop. This surprised me a little.

Take this analysis with a grain of salt… The source I used for this analysis might be a bit old (it’s actually from concurrent_queue_v2.h and .cpp) because the more recent concurrent_queue has a different API – there’s no pop() or pop_if_present(), just a try_pop() function in the latest class concurrent_queue interface. The old interface has been moved (possibly changed somewhat) to the concurrent_bounded_queue class. It appears that the newer concurrent_queues can be configured when the library is built to use OS synchronization objects instead of busy waits and polling.