I have a script that loads several hundred images, resize them and then composes a bigger image
Every time is started with a different set of images:
python myscript.py imageFolder/
Running it in a virtualenv with Pypy doesn’t show a noticeable speed gain (all run in ~8 seconds with mprofile, with the pypy version spending more time in PIL.resize and less in packages initialization).
It is because the JIT gives advantage only for long running processes?
If so I can convert the script to a daemon (but I fear of memory leaks).
From your description it appears that
PIL.resize()is the dominant operation. That function is written in C and not in Python. Therefore, I doubt you can expect PyPy to make much of a difference to your script.If you’re looking to speed things up, you could consider parallelizing the loading and resizing of the image across multiple cores. I don’t generally recommend using threads in Python, normally suggesting the
multiprocessingmodule instead. However, for this particular task multiple threads might actually be a better fit.