Question

A question yesterday about doubled-checked locking started a chain of thoughts that left me unsure about a simple situation. In the following code, is it possible to hit the printf of “No longer in sync”? In this simple example, the values would likely be on the same cache line, so I think it would be less likely (assuming the possibility is > 0% to begin with).

If the answer is, “No, it is not possible.”, then my follow-up question is, rather predictably: why not? Until I got my thoughts tangled and wrapped around the multi-threading axel yesterday, I assumed that the code would be safe. But now I am wondering what prevents a stale read from the cache for one of the variables pa or pb. And would it matter if pa, pb pointed to simple global integer variables rather than malloc’d memory? Does the WaitForSingleObject call provide a memory barrier? Or should the pointers be declared volatile? So many questions, so few sentences.

Update: I finally located information that does specifically say that functions that signal synchronization objects do use memory barriers. It should have been obvious, but I was having trouble finding a definitive answer. So I can once again delude myself into believing I understand it all.

int i1 = 0;
int i2 = 0;
int reads = 0;
int done = 0;
int *pa = NULL;
int *pb = NULL;
HANDLE hSync = NULL;

DWORD WriteThread( LPVOID pvParam )
{
   while( !done )
      {
      WaitForSingleObject( hSync, INFINITE );
      (*pa)++;
      (*pb)++;
      ReleaseSemaphore( hSync, 1, NULL );
      }
   return 0;
}

DWORD ReadThread( LPVOID pvParam )
{
   while( !done )
      {
      WaitForSingleObject( hSync, INFINITE );
      if ( *pa != *pb )
         {
         printf( "No longer in sync: %d, %d\n", *pa, *pb );
         exit( 1 );
         }
      ReleaseSemaphore( hSync, 1, NULL );
      reads++;
      }
   return 0;
}

int main( int argc, char* argv[] )
{
   DWORD dwID;

   // malloc'd memory
   pa = (int*)malloc( sizeof( int ));
   pb = (int*)malloc( sizeof( int ));

   // Is a simple global variable different?
   //pa = &i1;
   //pb = &i2;

   *pa = 0;
   *pb = 0;

   hSync = CreateSemaphore( NULL, 1, 1, NULL );
   CreateThread( NULL, 0, WriteThread, NULL, 0, &dwID );
   CreateThread( NULL, 0, ReadThread, NULL, 0, &dwID );

   while ( *pa < 1000000 )
      Sleep( 1 );
   done = 1;

   return 0;
}

Answer 1

It doesn’t matter where the memory lies, and if it were all about cache coherency, then declaring the variables volatile would do nothing to fix it. Volatile’s semantics are neither necessary nor sufficient for thread-safety; don’t use it!

At the C/C++ level, pa and pb may be cached in registers, but they will be considered stale after any function call. At the CPU level, all wait functions use barriers to make sure everything works as expected.