I’m using PHP to work with an API that limits me to 1 request every 10 seconds. The most results it can return in one go is 10.
That means to get 1000 results I need to make 100 calls which if done one after the other would take about 17mins so doing it ‘on the fly’ is not really an option.
If I need to get 1000 rows from the API, which would be the best way to go about it?
Is there any way I could get the API data ‘in the background’ so that when I need the info it’s already in my database? The API is only updated every 4 weeks so it would only need to synchronise once in that time period.
I’ve though about using a cron job to do this but i’m not sure how it would work considering how long the script would have to run for.
Using a cron job is a great way to store values from an API in a database. You need to use updated time fields for both the API and the local database. Check if the date of the last record update of the API is greater than the date of the last record update in your local database. If so, fetch the new data.
If you want to do this same process for the local database to the server, you can. This is a common operation to sync project flow and reduce API requests.