Question

I’m using Watin in C# console app to crawl websites, there are five console app running at the same time.
The reason I partly use Watin as crawler is because a few websites use javascript(or ajax) to set page content.

Following is the sample code to get a page comment count:

        Settings.Instance.MakeNewIeInstanceVisible = false;
        using (var browser = new IE(commentLink, true))
        {
            browser.Link(Find.ByUrl(commentLink)).WaitUntilExists(20);

            Span commentSpan = browser.Span("COUNT_TOTAL");

            if (commentSpan.Exists)
            {
                int commentCount;
                if (Int32.TryParse(commentSpan.InnerHtml, out commentCount))
                {
                    return commentCount;
                }
            }
        }

My problem is after running these 5 console app for a while (90 min), a lot of IE instances are stayed open (because of timeout or error or IE is busy), so the system is quite slow and need to reboot.

How do I change my code to prevent this thing happen and make my apps stay effecient?

Answer 1

I think, that in your sample code there is only one moment, when IE will not be close. Since you are inside using, even if exception inside of that using occurs, browser will be disposed, so everything is fine.

But in the moment of creating browser:

new IE(commentLink, true)

you are not inside using, and there is no magic try...catch.
Try this, and let me know if its helped:

Settings.Instance.MakeNewIeInstanceVisible = false;
using (var browser = new IE(true))
{
    browser.GoTo(commentLink);

    browser.Link(Find.ByUrl(commentLink)).WaitUntilExists(20);

    Span commentSpan = browser.Span("COUNT_TOTAL");

    if (commentSpan.Exists)
    {
        int commentCount;
        if (Int32.TryParse(commentSpan.InnerHtml, out commentCount))
        {
            return commentCount;
        }
    }
}