Question

I’m just starting to learn ASP.NET MVC. The website I’m working on requires the user to upload XML files which are loaded into a SQL database through NHibernate. This is working ok now with a small XML file, but once I tried a larger file I started having problems.

When I upload a larger file it takes a while for the data to be written to the database, so on the user’s end they get stuck “waiting for local host .. ” until the database write is complete and the Upload() action method returns a view.

Instead I’d like to set something up like most of the popular movie upload sites, where once the user uploads the file they are immediately sent to another page with a message that says something like “Your upload is being processed please come back later”.

I’ve done a little with multi-threading, but I hardly feel like I know what I am doing. Although, just now I seemed to have gotten my siteworking the way I want using System.Threading.ThreadPool.

Is using the thread pool like this the best way to approach this problem?

Here’s my Upload() action method.

public ActionResult Upload(int id)
{
    //check the type of request
    if (Request.RequestType == "GET")
        {
            //get GET show the upload form
            return View();
        }
        else if (Request.RequestType == "POST")
        {
            //if POST process uploaded file data
            if (Request.Files.Count == 1)
            {
                //copy the upload stream into a memory stream
                System.IO.MemoryStream memStream = new System.IO.MemoryStream();
                Request.Files[0].InputStream.CopyTo(memStream);

                //send the the memorystream, id, and username to a new Thread pool work item
                System.Threading.ThreadPool.QueueUserWorkItem(new System.Threading.WaitCallback(WriteUploadToDb), new WriteUploadToDbData(id, memStream,User.Identity.Name));

                //send the user to a page
                ViewBag.Message = "Your file is being processed. Please come back later...";
                return RedirectToAction("Reports", new { ID = id });
            }
            else
            {
                throw new Exception("Multiple report files uploaded!");
            }
        }
    }

And here’s my WriteUploadToDb() method which is run on the ThreadPool.

private static void WriteUploadToDb(object obj)
    {
        WriteUploadToDbData updateData = (WriteUploadToDbData)obj;

        //resets the stream position
        updateData.Data.Position = 0;

        //creates a new Nhibernate session
        using (var session = MvcApplication.SessionFactory.OpenSession())
        {
            using (var trans = session.BeginTransaction())
            {
                var proj = session.Get<Project>(updateData.ProjectID);

                //adds the xml data into the Project
                System.Xml.XmlDocument xmlDoc = new System.Xml.XmlDocument();
                xmlDoc.Load(updateData.Data);
                proj.AddReport(xmlDoc, updateData.UserName);

                //saves the project to the db
                session.Update(proj);
                trans.Commit();
            }
        }

        //disposes the memory stream
        updateData.Data.Dispose();
    }

Answer 1

Depends on the volume you’re doing this at.

For low to modest volume, kicking off an async thread to complete the work is fine. However, a given web server can only process so many threads (the thread pool has a finite, though configurable, size).

For high volume applications I prefer to have the web server only handle web requests and offload any async processing to another server. This can be accomplished e.g. using MSMQ or by writing to a log file that another server picks up and processes (the later is commonly used in very high volume applications since writing to a file is a very fast and very reliable process).

The reason I prefer to go this route is that I can scale to handle incoming web service / website requests independently from the processing of async work items, allowing for more control over the scale out.