Question

I have push task queues in GAE with rate 1/s. Not every task on that queue needs to be executed, but I can verify that only when it’s called. So whenever task is called I have “if (condition) execute else drop” statement.

Now my problem is I have too much tasks, and many of them will be dropped. Instead of just waiting for another task to be called in a next second I’d like to force my queue to call next task, so that second won’t be wasted.

What I’d like to ask – is it possible to force the queue to get another task right now instead of waiting? Or should I implement is on my own?

If it’s not clear, I’m sorry, I will try to put that in better words.

Thanks in advance!

Answer 1

You need to work with pull tasks, for every push task queue also a pull task. when your push task execute lease a batch of tasks from the pull queue and check the condition on them.
Next step will be to discard all the tasks that are not “valid” anymore and execute a single valid task. once your execute a valid task you return all the remaining tasks (lease their time to zero).

This flow should grantee at least one task per push task in case a valid task exists.