Our “Enforced Rate” dropped to 0.10/s even though the queue is defined to be 20/s for a Push Task Queue. We had a 2 hour backlog of tasks built up.
The documentation says:
The enforced rate may be decreased when your application returns a 503
HTTP response code, or if there are no instances able to execute a
request for an extended period of time.
There were no 503s (all 200s), yet we saw 0 (zero) instances running for the version processing that queue, which may explain the Enforce Rate drop. The target version of the application processing the queue is not the primary version, and the target version has no other source of requests than the Push Task Queue workload.
If the Enforced Rate had been dropped due to lack of instances, why did GAE not spin up new instances?
There were no budget limits or rate limits that explain the drop to 0 instances.
I’m answering this as the bug we’re starring http://code.google.com/p/googleappengine/issues/detail?id=7338