On my local dev environment, when an App Engine task that had been added to the the task queue hits an error, it is retried until successful. However, in the production environment, it’s not. What I THINK is happening is that, because I have a custom 500 handler in urls.py, all errors are caught by this and the 500 error never bubbles to the top.
Could this in fact be the reason that my tasks are not being retried? And if so, is there a way to prevent this? I only want errors to be caught by the handler500 view if the request is user-generated, not a backend task (for those, I want the error to bubble up and force a retry).
The task is retried by the Task Queue if it returns a non-200 status code. If your “custom 500 handler” is returning a 200 status code, then the Task Queue has no way of knowing the task failed, so it doesn’t retry it. You need to modify your handler to return the appropriate status code – which it should be doing regardless, including for user pages.