Question

I’m currently working on redesigning an application which is building a model from input data, and displaying that data to the user. The current system has a thread for building the model, a thread for building a visualization of the model, and a thread which displays the visualization. The problem I have is that pointers are passed between the modeling and visualization threads – in order to make the threads safe, all objects in the model have to have a mutex. This means there are thousands of mutexes active in the system, with lots of stalls as both threads contend for resources.

So, given that those three threads will exist, what is the best way to share data between the modeling and visualization threads in a manner which is efficient and thread safe? Some of the data structures are large and will change every cycle of the modeling thread, so I’m somewhat loath to make a copy of the data every pass.

EDIT:

The total latency through the system we’re hoping for is to have ~100ms from receiving a message to the display showing the change in the display. We would like it to be faster if possible, but more importantly we need it to be consistent – right now we see huge variability in the cycle times, due to the mutex contention. The data moving from modeling to visualization are dominated by 2D height maps – ~18000 cells worth of data. The actual number of cells updated in model update is significantly less though – perhaps only a few hundred.

Answer 1

I am a big fan of the message posting/message pump architecture. This the main method that MFC/Win32 provides to communicate data between threads. This architecture is event driven off of thread messages, so when the receiving thread is processing a thread message, it is processing data explicitly newed for the communication between threads (see example below).

You could implement this yourself, and localize the locking to each thread’s individual list of thread messages. So when one thread wants to message another thread you do roughly the following

PostThreadMessage(msg, void* param, int paramSize) {     lock(msgQueueLock);      // may wish to copy param     char paramCpy = malloc     msgQueue.Queue(msg, pparam, paramSize);       unlock(msgQueueLock); } 

then the main loop of any thread is just

// thread's msg pump while (1) {     // can also use condition var to wait for queue to change...     lock(msgQueueLock);     HandleMsgLocally(msgQueue.Deque())     unlock(msgQueueLock); } 

Anyway, getting back to MVC, if something changes in your model, it can post to your view to update a specific field as such:

// Well known msg name int msgName = MODEL_FIELD_A_UPDATED  ...  void Model::UpdateFieldA(int newVal) {     int* valToCommunicate = new int(newVal)     PostThreadMessage(MODEL_FIELD_A_UPDATED, valToCommunicate, sizeof(int)) }   ... void HandleMsgLocally(...void * param,) {     if (msg == MODEL_FIELD_A_UPDATED)     {        int* val = reinterpret_cast<int*>(param);        //... process param        delete val;     } } 

The advantages are you localize your locking. This is huge. Also so long as the param’s are understood to be explicitely new’d by the sender and deleted by the receiver, you don’t have to worry about accessing shared memory.

There are many disadvantages to this, latency being one. If you need to know right away that something changed, then you would need to actually make that shared data and think about the best locking scheme. If you truly need a global state that can get updated at any time from multiple directions, a singleton is fine in this case in my book. This design is really mostly suited for data that goes one direction, you can get into race conditions. You may also be able to, however, implement locking getters for inspection by one thread from another, but to set the value you have to post. There’s lots of variables to think about, but hopefully this might help you. Also, you can forget to delete the params in the message.

Update based on Edit Based on your info, depending on the amount of data, posting could very well work. Profiling your code would be important. If I were you I’d play with some proof of concept and switch to using condition variables to manage synchronicity. If you are on a Windows platform, definitely use their message pump.