Question

I have a singleton service class on IIS as part of a web application (the service to be a singleton for data caching reasons). Browser clients that make requests to the service can have one of three results:

1) There is data in the cache and the data is not expired (stale) – we return this data. Very fast.
2) The cached data is expired, but another request is already querying the database. We returned the cached data.
3) The cached data is expired and no requests are making queries to the DB. This request moves forward to make the query.

However, database queries that target stored procedures with the same name have to be queued (requirement).

Thus, I wrote this queue class that is designed to queue these queries and run them consecutively, instead of concurrently. These queue classes are created as needed and stored in a list in the singleton class. When a request moves to part (3), it finds the queue class that matches its stored procedure name and submits the request to the queue class. It then waits until the data is returned from the DB so it can service the HTML request.

Unfortunately, after a few hours with this code in place, the server process maxes at 100%.

I am not sure what the best way is to go about improving it, because multi-thrread coding is not my specialty.

The queue class code looks like this:

public ReportTable GetReportTable(ReportQuery query)
{
  lock (_queue)
  {
    _queue.Enqueue(query);
    Monitor.Pulse(_queue);
  }

  lock (_queue)
  {
    var firstQueryInQueue = _queue.Peek();
    while (_inUse || firstQueryInQueue == null || firstQueryInQueue.GetHashCode() != query.GetHashCode())
    {
      Monitor.Pulse(_queue);
      Monitor.Wait(_queue);
    }

    _inUse = true;
    firstQueryInQueue = _queue.Dequeue();
    var table = firstQueryInQueue.GetNewReportTable();
    _inUse = false;

    Monitor.Pulse(_queue);
    return table;
  }
}

Answer 1

So here’s what I did to fix it.

public ReportTable GetReportTable(ReportQuery query)
{
  lock (_queue)
  {
    _queue.Enqueue(query);
    Monitor.Pulse(_queue);
  }

  lock (_queue)
  {
    var firstQueryInQueue = _queue.Peek();
    while (_inUse || firstQueryInQueue == null || firstQueryInQueue.GetHashCode() != query.GetHashCode())
    {
      Monitor.Wait(_queue);
    }

    _inUse = true;
    firstQueryInQueue = _queue.Dequeue();
    var table = firstQueryInQueue.GetNewReportTable();
    _inUse = false;

    Monitor.Pulse(_queue);
    return table;
  }
}

The reason it didn’t work before was because of my lack of total understanding of Monitor.Wait() and Monitor.Pulse(). I was using Pulse() in a wrong place in the code. Fortunately, there’s a very good answer here that goes on to describe Wait() and Pulse() quite well.

The key is to Pulse() after the collection is changed, and give the subsequent queued threads a chance to test for the condition, which is: Is my query first in the queue? And is someone else already making a query? Failing that test, the thread calls Wait(), which puts it in the wait queue, and it doesn’t tie up processor cycles. When the thread that is in the front and is performing the query completes, it flips the _inUse flag to false and calls Pulse(), waking one of the next threads so it can check the condition.

After implementing this solution and watching the Task Manager for a day, I was happy to see 1% to 5% load on the server for several hours, and the CPU never climbed to 100% as before.

I’ve done a lot of reading on this and it seems that PulseAll() might be the better call in this scenario, but so far Pulse() is working and we haven’t encountered any issues.