We’re interested in using a push queue in GAE but one thing I can’t find is the window for recovery in the event of queue or appengine downtime.
For example, I have a push queue with a number of tasks on it. Some of these tasks get pulled off and are executing. Let’s say now the queue goes down (for whatever reason), while these tasks are executing, and then comes back up. What is the time window for the restoration of the queue? Is there a set time window of recovery?
There’s the possibility of these tasks that were pulled off the queue and executing now reappearing on the queue and having them execute again due to the restoration time window.
We’ve got idempotence considerations in our code, but it would be good to know if there are time window recovery strategies for the GAE Queue downtimes.
If I understand your question correctly, you’re worried that queues can go down in the sense that a knowledge of execution completion is lost for a particular eta range, and those tasks have to be re-executed.
This is not the way things work in the GAE task queue system. We track execution on a task by task basis. (We have to because the tasks need not be dispatched in strict eta order.) The queue doesn’t “go down” in the sense you’re referring to.
An individual task might execute successfully twice under the current system. When this happens (and it is very rare), there should be at least a minute between successive executions.
There is no time-window recover strategy you need to consider.