Question

I have developed an application that pulls X amount of records from my database per X amount of threads. Each thread then iterates the collections that are created and does some data validation for each record.
Once a record is validated / not validated by the application it is updated in the database as being valid / not valid. Records are only pulled by each thread if there are items in the database that have not been run through the application. There is a bit column to indicate if the application retrieved the data.
So, potentially, the system can run out of data depending on the number of threads and records per thread. I want the application to continue to check the database for any records that have not be run, then start the process of creating the threads, and finally validating the data.

Here is an example:
There are 50 records in the database
we are running 5 threads with 10 records per thread.
The application runs, the threads are created, the records are pulled and then processed. Now, the system is out of data. A user imports more data into the DB. The application, still looking to see if there are any records, sees that there are 5 new records in the database. It then starts the process all over to create the threads and process the records.

How can I have the system continue to look for data but allow the user to stop the system if need be. I tried using this:

while(RecordsFound <=0){
 …sleepcode
} ;
RunProcessMethod

But the winform locks, obviously, during the waiting period. I tried adding the wait logic to another thread, but was afraid that if I run the process method from that thread, via a delegate, things would get weird since I am creating additional threads inside that method.
Thoughts?

Answer 1

The easiest way to fix this is to use a notification mechanism instead of polling. That is once you’ve spawned off the threads to read data from the data base, make them responsible for notifying the UI when they are complete instead of having the UI wait for them to be complete.

The easiest way to do this is to pass in a delegate for the threads to call when they are complete with the set of records found. The UI can then update when the records are available

delegate void CallBack(List<Data> found);

void OnDataFound(List<Data> found) {
  // Get back on the UI thread
  if ( this.InvokeRequired ) { 
    this.Invoke( new CallBack(OnDataFound), new object[] { found } );
    return;
  } 

   // Update display
}