Question

I want to build an Azure application that has two worker roles and NO web roles. When the worker roles first start up I want ONLY ONE of the roles to do the following a single time:

• Download and parse a master file then enqueue multiple “child” tasks based on the
  contents of the master file
• Enqueue a single master file download “child” task to run the next day

Each of the “child” tasks would then be done by both of the workers until the task queue was exhausted. Think of the whole things as “priming the pump”

This sort of thing is really easy if I add the the first “master” task manually in a queue by calling a web role but seems to be really hard to do in an auto-start mode.

Any help in this regard would be greatly appreciated!

Thanks…..

Answer 1

One possibility: instead of calling a web role, just load the queue directly. (It sounds like this is the sort of application you’ll want to automatically spin up to do some work and then shut down again… if you’re automating that, it should be trivial to also automate loading the queue.)

A (perhaps) better option: Use some sort of locking mechanism to make sure only one worker instance does the initialization work. One way to do this is to try to create the queue (or a blob, or an entity in a table). If it already exists, then the other instance is handling initialization. If the create succeeds, then it’s this instance’s job.

Note that it’s always better to use a lease than a lock, in case the instance that’s doing the initialization fails. Consider using a timeout (e.g. storing a timestamp in table storage or in the metadata of the blob or in the name of the queue…).