According to https://developers.google.com/appengine/docs/python/config/cron cron jobs can run for 10 minutes. However, when I try and test it by going to the url for the cron job when signed in as an admin, it times out with a DeadlineExceededError. Best I can tell this happens about 30 seconds in, which is the non-cron limit for requests. Do I need to do something special to test it with the cron rules versus the normal limits?
Here’s what I’m doing:
- Going to the url for the cron job
- This calls my handler which calls a single function in my py script
- This function does a database call to google’s cloud sql and loops through the resulting rows, calling a function on each row that use’s ebay’s api to get some data
- The data from the ebay api call is stored in an array to all be written back to the database after all the calls are done.
- Once the loop is done, it writes the data to the database and returns back to the handler
- The handler prints a done message
It always has issues during the looping ebay api calls. It’s something like 500 api calls that have to be made in the loop.
Any idea why I’m not getting the full 10 minutes for this?
Edit: I can post actual code if you think it would help, but I’m assuming it’s a process that I’m doing wrong, rather than an error in the code since it works just fine if I limit the query to about 60 api calls.
The way GAE executes a cron job allows it to run for 10 min. This is probably done (i’m just guessing here) through checking the user-agent, IP address, or some other method. Just because you setup a cron job to hit a URL in your application doesn’t mean a standard HTTP request from your browser will allow it to run for 10 minutes.
The way to test if the job works is to do so on the local dev server where there is no limit. Or wait until your cron job executes and check the logs for any errors.
Hope this helps!