Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
The latest Google App Engine release supports a new Task Queue API in Python. I was comparing the capabilities of this API vs the already existing Cron service. For background jobs that are not user-initiated, such as grabbing an RSS feed and parsing it on a daily interval. Can and should the Task Queue API be used for non-user initiated requests such as this?
I’d say “sort of”. The things to remember about task queues are:
1) a limit of operations per minute/hour/day is not the same as repeating something at regular intervals. Even with the token bucket size set to 1, I don’t think you’re guaranteed that those repetitions will be evenly spaced. It depends how serious they are when they say the queue is implemented as a token bucket, and whether that statement is supposed to be a guaranteed part of the interface. This being labs, nothing is guaranteed yet.
2) if a task fails then it’s requeued. If a cron job fails, then it’s logged and not retried until it’s due again. So a cron job doesn’t behave the same way either as a task which adds a copy of itself and then refreshes your feed, or as a task which refreshes your feed and then adds a copy of itself.
It may well be possible to mock up cron jobs using tasks, but I doubt it’s worth it. If you’re trying to work around a cron job which takes more than 30 seconds to run (or hits any other request limit), then you can split the work up into pieces, and have a cron job which adds all the pieces to a task queue. There was some talk (in the GAE blog?) about asynchronous urlfetch, which might be the ultimate best way of updating RSS feeds.