Question

I have about 5000 html files in a folder. I need to loop through them, open, grab say 10 values using xpath, close, and store in (SQL Server) DB.

What is the easiest way to do read the xpath values using .Net?

The xpaths should be pretty stable.

Please provide example code to read one value, say /html/head/title/text()

Thanks

Answer 1

I think you should look into the HTML Agility Pack. It is an HTML parser rather than an XML parser, and is better for this task. If there is anything that doesn’t agree with the XML being parsed then the parser will throw and exception. Using an HTML parser gives you a bit more leeway with the input files.

Example showing how to do something with all HREF (link) attributes:

 HtmlDocument doc = new HtmlDocument();
 doc.Load("file.htm");
 foreach(HtmlNode link in doc.DocumentElement.SelectNodes("//a[@href"])
 {
    HtmlAttribute att = link["href"];
    att.Value = FixLink(att);
 }

I’m not near a compiler but the example you want is something like:

string title = doc.DocumentNode.SelectSingleNode("//title").InnerText;