Question

I am parsing a webpage for http links by first parsing out all the anchored tags, then parsing out the href tags, then running a regex to remove all tags that aren’t independent links (like href=”/img/link.php”). The following code works correctly, but also appends lots of blank lines in between the parsed links.

while (parse.ParseNext("a", out tag))
{
    string value;
    //A REGEX value, this one finds proper http address'
    Regex regexObj = new Regex(@"^http\://[a-zA-Z0-9\-\.]+\.[a-zA-Z]{2,3}(/\S*)?$");
    if (tag.Attributes.TryGetValue("href", out value))
    {
        string value2;
        //Start finding matches...
        Match matchResults = regexObj.Match(value);
        value2 = matchResults.Value;
            lstPages.AppendText(value2 + "\r\n");                        
    }
}

To fix this, I added the following code and it works to clean up the list:

if (value2 != "")
{
    lstPages.AppendText(value2 + "\r\n");
}

However, I

1. Don’t believe this is the most efficient way to go about this and
2. Still don’t understand where the != "" lines come from.

My actual question is on both of these but more for issue #2, as I would like to learn why I receive these results, but also if there is a more efficient method for this.

Answer 1

The reason you are getting an empty string in value2 is that matchResults.Value == "" if the regular expression fails to match. Instead of checking if value2 != "", you could directly check matchResults.Success to see if the regular expression matched. You’re basically doing that, anyway, since your particular regular expression would never match an empty string, but checking matchResults.Success would be more straightforward.

Another thing to consider is that it’s not necessary to create the Regex object every iteration of your loop. Here are the modifications I suggest:

//A REGEX value, this one finds proper http address'
Regex regexObj = new Regex(@"^http\://[a-zA-Z0-9\-\.]+\.[a-zA-Z]{2,3}(/\S*)?$");
while (parse.ParseNext("a", out tag))
{
    string value;

    if (tag.Attributes.TryGetValue("href", out value))
    {
        string value2;
        //Start finding matches...
        Match matchResult = regexObj.Match(value);
        if (matchResult.Success)
        {
            value2 = matchResult.Value;
            lstPages.AppendText(value2 + "\r\n");
        }
    }
}