I’ve written a PHP script which performs web scraping from one site and parse input for my website.
The script is driven by cronjob periodically, and everything is hosted in a shared web-server.
The problem is: my script terminates several times a day, with no error message and in random location in code each time.
The script is long, performing 2 HTTP Gets and 4 HTTP Posts to other country website, each HTTP request takes ~3 seconds to complete; it also writes to files and r/w to/from MySql database.
I’m stuck on it after trying the following things:
1) Talking with my hosting support (IxWebHosting) – they just wasted my time, denying their responsibility and advised me to limit the cronjob periodicy to 5 minute rate maximum (before it was 3 minutes interval, however it didn’t change anything.)
2) Instead of running from cronjob context, I’ve switched to the following method:
a. Cronjob calls a “loader PHP script” every 5 minutes.
b. The “loader PHP script” calls the real PHP script using HTTP Get and terminates before waiting for an answer.
c. The real PHP script perform its ~20 seconds job (here is where the program terminates in random location).
3) Put some log file timestamp writing in many places along the code in order to see where program terminated each run – this showed me the program terminates everywhere in the code.
4) In order to prove it’s not my code fault I’ve performed the following test:
a. Cronjob calls another loader PHP script.
b. The PHP script performs HTTP request to a different testing-purpose PHP script and terminates without waiting for response.
c. The 2nd PHP script will perform dummy 20 seconds task: sleep for a second and write timestamp into log file for 20 times.
Result: the test succeeded! the 2nd program didn’t fail… which means it has something to do with my code and the webserver I’m running at – however since it fails everytime in different place and only ~10 times a day (from 288 times it runs a day) then I can’t tell where it is (and no error message of PHP).
Thanks in advance, sorry for long description – I’ll be happy to provide more details upon request.
Are you logging the actual process, rather than writing logs from within the process ? e.g. does your cron job look like:
This will catch the stdout/sterr of the process itself. It may also be worth invoking your script from a parent shell script, and that can catch the PHP process exiting, and dump out the exit code (which would indicate a core dump, a signal being caught etc.). See here for more info.