Question

While I’m familiar with concurrent programming concepts such as mutexes and semaphores, I have never understood how they are implemented at the assembly language level.

I imagine there being a set of memory “flags” saying:

• lock A is held by thread 1
• lock B is held by thread 3
• lock C is not held by any thread
• etc

But how is access to these flags synchronized between threads? Something like this naive example would only create a race condition:

  mov edx, [myThreadId]
wait:
  cmp [lock], 0
  jne wait
  mov [lock], edx
  ; I wanted an exclusive lock but the above 
  ; three instructions are not an atomic operation :(

Answer 1

• In practice, these tend to be implemented with CAS and LL/SC.
  (…and some spinning before giving up the time slice of the thread – usually by calling into a kernel function that switches context.)
• If you only need a spinlock, wikipedia gives you an example which trades CAS for lock prefixed xchg on x86/x64. So in a strict sense, a CAS is not needed for crafting a spinlock – but some kind of atomicity is still required. In this case, it makes use of an atomic operation that can write a register to memory and return the previous contents of that memory slot in a single step. (To clarify a bit more: the lock prefix asserts the #LOCK signal that ensures that the current CPU has exclusive access to the memory. On todays CPUs it is not necessarily carried out this way, but the effect is the same. By using xchg we make sure that we will not get preempted somewhere between reading and writing, since instructions will not be interrupted half-way. So if we had an imaginary lock mov reg0, mem / lock mov mem, reg1 pair (which we don’t), that would not quite be the same – it could be preempted just between the two movs.)
• On current architectures, as pointed out in the comments, you mostly end up using the atomic primitives of the CPU and the coherency protocols provided by the memory subsystem.
• For this reason, you not only have to use these primitives, but also account for the cache/memory coherency guaranteed by the architecture.
• There may be implementation nuances as well. Considering e.g. a spinlock:
  • instead of a naive implementation, you should probably use e.g. a TTAS spin-lock with some exponential backoff,
  • on a Hyper-Threaded CPU, you should probably issue pause instructions that serve as hints that you’re spinning – so that the core you are running on can do something useful during this
  • you should really give up on spinning and yield control to other threads after a while
  • etc…
• this is still user mode – if you are writing a kernel, you might have some other tools that you can use as well (since you are the one that schedules threads and handles/enables/disables interrupts).