Question

One class XYZ has 3 member variable x, y, z. There is vector of [N] XYZ objects.

There are 3 threads A, B, C, they can access any objects in the vector and any member variable of that object.

class XYZ
{
public:
double x;
double y;
double z;
};

std::vector with N elements, where N is fixed throughout the program.

How to design the inter-thread communication to achieve thread-safety and maximum efficiency, i.e., minimum blocking.

Here is some of my thinking process, please correct me if I am wrong.

1. Divide vector in smaller vectors and encapsulate into each thread
   class, then use a message queue to pass data around. The problems
   are all 3 threads can access anywhere of the vector and objects
   member, therefore it is difficult to sub-divide and encapsulate.
   Message queue itself needs blocking, i.e., reader needs to be
   blocked when sender is adding to the queue.
2. Use atomic library to make access atomic, therefore avoiding
   blocking. The problem is atomic is OS dependent, i.e., some
   operation is considered atomic under Linux may not be atomic under
   Windows.
3. Mutex, add 3 mutex objects for each member variable, e.g., mutex_x,
   mutex_y, mutex_z. However, the problem is that mutex is noncopyable,

i.e., if we have a class like,

class XYZ_mutex
{
public:
double x;
double y;
double z;
boost::mutex mutex_x;
boost::mutex mutex_y;
boost::mutex mutex_z;
};

We can NOT have a vector of XYZ_mutex, because .push_back() is a copy constructor.

Thanks.

Answer 1

You need to consider what usage patterns and what consistency requirements your program has.

First and most important usage pattern is whether any of the threads needs to modify these structures. If not, you don’t actually need any locking – just make sure the structure is populated before the threads start reading it.

If the threads do have to modify the structure, you need to consider consistency. You need to ask yourself, if there are any constraints, other than modifying an individual double value, that limit when one thread can see changes made by another thread.

Once you define that, start thinking if data can be split between threads in any way to reduce conflict between threads – even if it can’t be completely eliminated, the answer the question whether it’s better to keep three vectors of doubles, a vector of XYZ or some other organisation depenst on whether there will be some threads that access certain XYZ objects more often, or if there will be threads that access x members more often, while others access ys or zs more often etc.

If you can’t say anything about the probability of accessing any member of any instance by any thread, it’s hard to say how to best organise them. It might even be a good idea to place them on the heap so that they land in separate cache lines.

Overall, probably the best suggestion is to start by putting them in any data structure that’s best suited for your needs (vector, map, set etc.), using a single mutex to synchronise the whole thing, writing your program and then testing if you experience a bottleneck.