Home/ Questions/Q 3223584
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-17T16:07:26+00:00

I have a special situation where I need to query a service to get

  • 0

I have a special situation where I need to query a service to get information for n users at a specific time each day. The problem is that if I do this all at once it will put the service offline / crash it.

So to overcome this it would be better to run this for x number of users every 10 seconds or so until x = n and then stop.

I could setup 1 cron script that runs daily and another that runs every 10 seconds. The daily script would set a value in the DB to 1 (‘start query’) for example (default would be 0 for off), where then the second script (run every 10 seconds) checks this database value for 1. Upon finding the setting set to true it then iterates through the users querying the service x users at a time and incrementing another column in the same DB table to keep track of where in the list it is at.

The problem with this solution (well according to me) is that the 2nd script that runs every 10secs has to query the DB each time to find out if the ‘start query’ setting is set to 1. This can be quite processor heavy. Does anyone have a better solution?

NB: Code is written in PHP – cannot use sleep due to max execution time of php scripts on server

I could equally do this in python, is there a max execution for limit on cgi scripts?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Why not just have one cron script that runs daily, and just calls the other script every ten seconds until the other script says it’s done (e.g. returns a value that says “nothing more to process”)?

    Or one script that runs daily and just runs through the chunks of users in a loop, sleep()ing every X users…

    Bear in mind that when using the CLI version of PHP you’re not constrained by execution time limits, and you can interact with cron, and other shell scripts quite nicely, using standard error and output streams, etc.

    For example, for the first approach, you don’t even have to use PHP for the “controlling” script — a very simple shell script that just loops, calling your PHP script and then sleeping, and checking for a “success” return value (as returned by the PHP script’s exit() function) would be fine.

    • 0