I have a function that needs to go over around 20K rows from an array, and apply an external script to each. This is a slow process, as PHP is waiting for the script to be executed before continuing with the next row.
In order to make this process faster I was thinking on running the function in different parts, at the same time. So, for example, rows 0 to 2000 as one function, 2001 to 4000 on another one, and so on. How can I do this in a neat way? I could make different cron jobs, one for each function with different params: myFunction(0, 2000), then another cron job with myFunction(2001, 4000), etc. but that doesn’t seem too clean. What’s a good way of doing this?
If you’d like to execute parallel tasks in PHP, I would consider using Gearman. Another approach would be to use pcntl_fork(), but I’d prefer actual workers when it’s task based.