Question

Let’s assume we have some task, that could be divided into independent subtasks and we want to process these tasks in parallel on the same machine.

I read about multithreading and ran into this post, which describes GlobalInterpreterLocks. Since I do not understand fully how processes are handled under the hood, I got to ask:

Putting aside the gain of threading: Is Multithreading (in my case in python) effectively the same as calling a script multiple times?

I hope this question does not lead to far and its answer is understandable for someone whose knowledge about the things happening on the low levels of a computer are sparse. Thanks for any enlightening in this matter.

Answer 1

Is Multithreading (in my case in python) effectivle the same as calling a script multiple times?

In a word, no.

Due to the GIL, in Python it is far easier to achieve true parallelism by using multiple processes than it is by using multiple threads. Calling the script multiple times (presumably with different arguments) is an example of using multiple processes. The multiprocessing module is another way to achieve parallelism by using multiple processes. Both are likely to give better performance than using threads.

If I were you, I’d probably consider multiprocessing as the first choice for distributing work across cores.