I’m looking at leveraging dynamic backends to do some large scale push taskqueue-style jobs that require a little longer than ten minutes to crunch. The ability for dynamic backends to come up on-demand and “autoscale” based upon my taskqueue volume is extremely attractive.
When configuring these queues with a ‘target’ of ‘None’, this causes these queues to process using front-end instances. Since I’m using Python27 with threadsafe, this allows my task to be handled by multiple threads on fewer instances. My app does a lot of URLFetch-ing and time.sleep-ing, so I feel that I’ve got a lot of CPU-power to burn off.
Ideally, I’d like to spin up 20 backend instances of the same backend, but run as many ‘requests’ as possible simultaneously.
The question is: Do backends behave this way?
Yes, Backends should behave the same way.