Question

In general what is the relation between CPU usage and number of threads in a program.
Assumptions:

• Multi-core CPU
• Threads do the exact same job (assume they fetch identical work items from a queue and process them)

Answer 1

It depends on the nature of the application.

• An application that mostly do calculations – a ratio of 1 thread per
  core is a reasonable decision, since you don’t want to spawn too many threads due to overhead, and you want to take advantage of all your cores.
• An application that mostly do IO operations (like http requests) can spawn much more threads then the #cores and still increase efficiency, since the bottleneck is the waiting time per IO request, and you want to gain as much information as possible in each time you need to wait.

That said, the CPU-usage you are going to get is still dependent on many factors (IO, synchronization, non parallel parts in your program).

If you are interested in the speed the application will take – always remember Amdahl’s law, which gives you a strict bound on the time (speed-up) your application is going to take, even when having infinite number of working cores.