Question

I’ve got a handler (list.ashx for example) that has a method that retrieves a large dataset, then grabs only the records that will be shown on any given “page” of data. We are allowing the users to do sorting on these results. So, on any given page run, I will be retrieving a dataset that I just got a few seconds/minutes ago, but reordering them, or showing the next page of data, etc.

My point is that my dataset really hasn’t changed. Normally, the dataset would be stuck into the viewstate of a page, but since I’m using a handler, I don’t have that convenience. At least I don’t think so.

So, what is a common way to store the viewstate associated with a current user’s given page when using a handler? Is there a way to take the dataset, encode it somehow and send that back to the user, and then on the next call, pass it back and then rehydrate a dataset from those bits?

I don’t think Session would be a good place to store it since we might have 1000 users all viewing different datasets of different data, and that could bring the server to its knees. At least I think so.

Does anyone have any experience with this kind of situation, and can you give me any advice?

Answer 1

In this situation I would use a cache with some type of user and query info as the key. The reason being is you say it is a large dataset. Right there is something you don’t want to be pushing up and down the pipe constantly. Remember your server still has to received the data if it is in ViewState and handle it. I would do something like this which would cache it for a specific user and have a short expiry:

public DataSet GetSomeData(string user, string query, string sort)
{
    // You could make the key just based on the query params but figured
    // you would want the user in there as well.
    // You could user just the user if you want to limit it to one cached item
    // per user too.
    string key = string.Format("{0}:{1}", user, query);

    DataSet ds = HttpContext.Current.Cache[key] as DataSet;
    if (ds == null)
    {
        // Need to reload or get the data
        ds = LoadMyData(query);

        // Now store it and make the expiry short so it doesn't bog up your server
        // needlessly... worst case you have to retrieve it again because the data
        // has expired.
        HttpContext.Current.Cache.Insert(key, ds, null,
            DateTime.UtcNow.AddMinutes(yourTimeout), 
            System.Web.Caching.Cache.NoSlidingExpiration);
    }

    // Perform the sort or leave as default sorting and return
    return (string.IsNullOrEmpty(sort) ? ds : sortSortMyDataSet(ds, sort));
}

When you say 1000’s of users, does that mean concurrent users? If your expiration time was 1 minute how many concurrent users would make that call in a minute and require sorting. I think offloading the data to something like similar to ViewState is just trading some cache memory for bandwidth and processing load of larget requests back and forth. The less you have to transmit back and forth the better in my opinion.