Question

I’m sorry I don’t seem to get intel’s TBB it seems great & supported but I can’t wrap my head around how to use it since I guess I’m not used to thinking of parallelism in terms of tasks but instead saw it as threads.

My current workload has a job that sends work to a queue to keep processing(think of a recursion but instead of calling itself it sends work to a queue). The way I got this working in Java was to create a concurrent queue(non-blocking queue) and threadpoolexecutor that worked the queue/send work back to it. But now I’m trying to do something similar in c++, I found TBB can create pools but its approach is very different(Java threads seem to just keep working as long as their is work in the queue but TBB seems to break the task down at the beginning).

Here’s a simple Java example of what I do(before this I set how many threads I want,etc..):

static class DoWork implements Callable<Void> {
    // queue with contexts to process
    private Queue<int> contexts;

    DoWork(Context request) {
        contexts = new ArrayDeque<int>();
        contexts.add(request);
    }

    public Void call() {
        while(!contexts.isEmpty()) {
            //do work 
            contexts.add(new int(data)); //if needs to be send back to the queue to do more work
        }
    }
}

I sure its possible to do this in TBB, but I’m just not sure how because it seems to break up my work at the time I send it. So if it there’s 2 items in the queue it may only launch 2 threads but won’t grow as more work comes in(even if I have 8 cores).

Can someone help me understand how to achieve my tasks and also maybe suggest a better way to think about TBB coming from using Java’s threading environment(also I have no allegiance to TBB, so if there’s something easier/better then I’m happy to learn it. I just don’t like c++ threadpool because it doesn’t seem actively developed)?

Answer 1

The approach based on having a queue of items for parallel processing, where each thread just pops one item from the queue and proceeds (and possibly adds a new item to the end of the queue at some point) is fundamentally wrong since it limits parallelism of the application. The queue becomes a single point of synchronization, and threads need to wait in order to get access to the next item to process. In practice this approach works when tasks (each items’ processing job) are quite large and take different times to complete, allowing queue to be less contended as opposed to when (most of the) threads finish at the same time and come to the queue for their next items to process.

If you’re writing a somewhat reusable piece of code you can not guarantee that tasks are either large enough or that they vary in size (time to execute).

I assume that you application scales, which means that you start with some significant number of items (much larger than the number of threads) in your queue and while threads do the processing they add enough tasks to the end, so that there’s enough job for everyone until application finishes.

If that’s the case I would rather suggested that you kept two thread-safe vectors of your items (TBB’s concurrent_vectors for instance) for interchangeability. You start with one vector (your initial set of items) and you enque() a task (I think it’s described somewhere in chapter 12 of the TBB reference manual), which executes a parallel_for over the initial vector of items. While the first batch is being processed you would push_back the new items onto the second concurrent_vector and when you’re done with the first one you enque() a task with a parallel_for over the second vector and start pushing new items back into the first one. You can try and overlap parallel processing of items better by having three vectors instead of two and gradually moving between them while there’s still enough work for all thread to be kept busy.