Home/ Questions/Q 7869547
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-03T01:21:30+00:00

My understanding of the differences between CPUs and GPUs is that the GPUs are

  • 0

My understanding of the differences between CPUs and GPUs is that the GPUs are not general purpose processors such that if a video card contains 10 GPUs, each GPU actual share the same program pointer and to optimize parallelism on the GPU I need to ensure each GPU is actually running the same code.

Synchronisation is not a problem on the same card since each GPU is physically running in parallel so they should all complete at the same time.

My question is, how does this work on multiple cards? At the speed at which they operate at, doesn’t the hardware make a slight difference in execution times such that a calculation on one GPU on one card may end quicker or slower than the same calculation on another GPU on another card?

thanks

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Synchronisation is not a problem on the same card since each GPU is physically running in parallel so they should all complete at the same time.

    This is not true. Different threads on a GPU may complete at different times due to differences in memory access latency, for example. That is why there are synchronization primitives in OpenCL such as the barrier command. You can never assume that your threads are running precisely in parallel.

    The same is true for multiple GPUs. There is no guarantee that they are in sync, so you will need to rely on API calls such as clFinish to explicitly synchronize their work.

    • 0