I’m using an app on GAE to receive emails generated by PBS pro running on a cluster. The app parses through the emails for statistics on the cluster jobs and generates a simple report.
The problem is that when a large set of jobs are started on the cluster, the app can get hit with 1000+ emails in a matter of seconds. A huge number of instances are spun up to handle the requests. I can control this by setting the allowed latency to be on the order of several seconds, but this can create visible latency when serving the front page of the app that has the report.
Is there a way to set acceptable latency for each separate URL so that
/_ah/mail/user@myapp.appspotmail.com
can have high latency without causing more instances to launch, but if any of the other URLs are causing high latency more instances will be launched?
–Andrew
As far as I know, there is no way to set the max latency in the admin console, based on routes. However, what you could do, is once you receive your email, give the text of the email to a taskqueue task. These can be rate-limited to a very granular detail. If you’re using python, you can use the deferred module, so you don’t have to create separate handlers for these taskqueue tasks. I have no experience using java on GAE, so I don’t know if there’s a java equivalent of that.