Question

I have a table of urls which I need to loop through, download each file, update the table and return the results. I want to run up to 10 downloads at a time so am thinking about using delegates as follows:

DataTable photos;
bool scanning = false,
    complete = false;
int rowCount = 0;

public delegate int downloadFileDelegate();

public void page_load(){

    photos = Database.getData...        

    downloadFileDelegate d = downloadFile;

    d.BeginInvoke(downloadFileComplete, d);
    d.BeginInvoke(downloadFileComplete, d);
    d.BeginInvoke(downloadFileComplete, d);
    d.BeginInvoke(downloadFileComplete, d);
    d.BeginInvoke(downloadFileComplete, d);

    while(!complete){}

    //handle results...

}

int downloadFile(){

    while(scanning){} scanning = true;

    DataRow r;

    for (int ii = 0; ii < rowCount; ii++) {

        r = photos.Rows[ii];

        if ((string)r["status"] == "ready"){

            r["status"] = "running";

            scanning = false; return ii;                

        }

        if ((string)r["status"] == "running"){

            scanning = false; return -2;                

        }

    }

    scanning = false; return -1;        

}

void downloadFileComplete(IAsyncResult ar){

    if (ar == null){ return; }

    downloadFileDelegate d = (downloadFileDelegate)ar.AsyncState;

    int i = d.EndInvoke(ar);

    if (i == -1){ complete = true; return; }      


    //download file...

    //update row
    DataRow r = photos.Rows[i];

    r["status"] = "complete";

    //invoke delegate again
    d.BeginInvoke(downloadFileComplete, d);

}

However when I run this it takes the same amount of time to run 5 as it does 1. I was expecting it to take 5 times faster.

Any ideas?

Answer 1

You look like you’re trying to use lockless syncronization (using while(scanning) to check a boolean that’s set at the start of the function and reset at the end), but all this succeeds in doing is only running one retrieval at a time.

1. Is there a reason that you don’t want them to run concurrently? This seems like the entire point of your exercise. I can’t see a reason for this, so I’d lose the scanning flag (and associated logic) entirely.
2. If you’re going to take this approach, your boolean flags need to be declared as volatile (otherwise their reads could be cached and you could wait endlessly)
3. Your data update operations (updating the value in the DataRow) must take place on the UI thread. You’ll have to wrap these operations up in a Control.Invoke or Control.BeginInvoke call, otherwise you’ll be interacting with the control across thread boundaries.
4. BeginInvoke returns an AsyncWaitHandle. Use this for your logic that will take place when the operations are all complete. Something like this

–

WaitHandle[] handles = new WaitHandle[]
{
    d.BeginInvoke(...),
    d.BeginInvoke(...),
    d.BeginInvoke(...),
    d.BeginInvoke(...),
    d.BeginInvoke(...)
}

 WaitHandle.WaitAll(handles);

This will cause the calling thread to block until all of the operations are complete.