Question

I want to parse the HTML of a website in my C# program.

First, I use the SGMLReader DLL to convert the HTML to XML. I use the following method for this:

XmlDocument FromHtml(TextReader reader)
{
    // setup SGMLReader
    Sgml.SgmlReader sgmlReader = new Sgml.SgmlReader();
    sgmlReader.DocType = "HTML";
    sgmlReader.WhitespaceHandling = WhitespaceHandling.None;
    sgmlReader.CaseFolding = Sgml.CaseFolding.ToLower;
    sgmlReader.InputStream = reader;

    // create document
    XmlDocument doc = new XmlDocument();
    doc.PreserveWhitespace = true;
    doc.XmlResolver = null;
    doc.Load(sgmlReader);
    return doc;
}

Next, I read a website and try to look for the header node:

var client = new WebClient();
var xmlDoc = FromHtml(new StringReader(client.DownloadString(@"http://www.switchonthecode.com")));
var result = xmlDoc.DocumentElement.SelectNodes("head");

However, this query gives an empty result (count == 0). But when I inspect the results view of xmlDoc.DocumentElement, I see the following:

Any idea’s why there are no results? Note that when I try another site, like http://www.google.com, it works.

Answer 1

You need to select using the namespace explicitly, see this question.

XmlNamespaceManager manager = new XmlNamespaceManager(doc.NameTable);
manager.AddNamespace("ns", "http://www.w3.org/1999/xhtml");

doc.DocumentElement.SelectNodes("ns:head", manager);