Question

Google app engine task queues have configuration as (example)

  <queue>
    <name>mail-queue</name>
    <rate>5/m</rate>
    <bucket-size>10</bucket-size>
  </queue>

Here, what does the ‘bucket-size’ mean? I could not find a comprehensive documentation about this in google app engine documentation.

Does specifying this as 10 means that if 100 tasks are queued at an instant only 10 of those will be put in the queue and rest will be ignored?

Answer 1

bucket-size is perfectly described here:

Limits the burstiness of the queue’s processing, i.e. a higher bucket size allows bigger spikes in the queue’s execution rate. For example, consider a queue with a rate of 5/s and a bucket size of 10. If that queue has been inactive for some time (allowing its "token bucket" to fill up), and 20 tasks are suddenly enqueued, it will be allowed to execute 10 tasks immediately. But in the following second, only 5 more tasks will be able to be executed because the token bucket has been depleted and is refilling at the specified rate of 5/s.

If no bucket_size is specified for a queue, the default value is 5.

For your case it means that if 100 messages are queued, only ten are directly being executed and another 5 every next minute. You won’t loose any messages, but they will queue up if your bucket-size and rate is too low.