Question

i m using C# and Agility Pack to scrape a website but the result i m getting differs from what i m seeing in firebug. I suppose this is cause the site is using some Ajax.

// The HtmlWeb class is a utility class to get the HTML over HTTP
HtmlWeb htmlWeb = new HtmlWeb();
// Creates an HtmlDocument object from an URL
HtmlAgilityPack.HtmlDocument document = htmlWeb.Load("http://www.saxobank.com/market-insight/saxotools/forex-open-positions");
// Targets a specific node
HtmlNode someNode = document.GetElementbyId("href");
// If there is no node with that Id, someNode will be null
richTextBox1.Text = document.DocumentNode.OuterHtml;

Could someone advise on the code on how to do this correctly cause what i m getting back is just plain html. What i m looking for is for a

div id= "ctl00_MainContent_PositionRatios1_canvasContainer"

Any ideas?

Answer 1

If you have the ID, then use it – href is the name of an attribute, not the id.

HtmlNode someNode = 
    document.GetElementbyId("ctl00_MainContent_PositionRatios1_canvasContainer");

This will be the div with that id.

You can then select any a child node and its href attribute of that node using:

var href = someNode.SelectNodes("//a")[0].Attributes["href"].Value;