Question

in a project I’m working on, a client wants to combine RSS feeds into their sites news items.

The site already has a news feed populated by NewsItems pulled from the database.

My idea is to consume the rss feeds periodically (every 20 minutes or so), but I’m unsure what effect this would have on a client attempting to view the news page at the time the feed is being updated.

My idea is to use ThreadPool.QueueUserWorkItem to consume the feed, process it, and add new items to the database/lucene index/add tags etc, but my experience of async programming is pretty much zero.

Would QueueUserWorkItem spawn a new thread to do the work without holding up the clients thread? What would happen when the update ends?

I was thinking of doing something like this:

public class ExternalRSS
{
    private readonly SettingsDTO Settings;

    public ExternalRSS()
    {
        Settings = Application.FetchSettings();

        RunAsynchronously(ProcessFeed);
    }

    public static void RunAsynchronously(Action method)
    {
        ThreadPool.QueueUserWorkItem(i =>
        {
            try
            {
                method();
            }
            catch (ThreadAbortException ex) { /*log*/}
            catch (Exception ex){/*log*/}
        });
    } 

    private void ProcessFeed()
    {
        if(Settings != null)
        {
            using (var reader = XmlReader.Create(Settings.FacebookRssFeed))
            {
                var rssData = SyndicationFeed.Load(reader);

                //do processing etc
            }
        }
    }
}

Am I over complicating the problem?

Any advice would be welcomed!

Answer 1

This IMHO is not a task that should be done inside an ASP.NET MVC application. I would use a separate dedicated Windows Service that will periodically query the remote RSS feed and update the database. Or a simple console application scheduled to run at regular intervals with the Windows Task Scheduler. The important part is when writing to the database: make sure it is performed inside a SQL transaction. This way the client MVC application won’t see corrupt data when reading it.

I would also recommend you reading the following blog post in which Phil Haack explains in details why implementing such recurrent background tasks in ASP.NET applications is a very bad idea.