Question

I try to run the following code:

 public void Init(Url rootUrl)
        {
             var web = new HtmlWeb();
            this.doc = web.Load(rootUrl.Value);
        }

with the following param:

{<System.Security.Policy.Url version="1">
<Url>http://localhost:85/HCM/HCM.html</Url>
</System.Security.Policy.Url>
}

and get the following exception:
Object reference not set to an instance of an object.

Stack Trace:

   at HtmlAgilityPack.HtmlDocument.ReadDocumentEncoding(HtmlNode node) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlDocument.cs:line 1916
   at HtmlAgilityPack.HtmlDocument.PushNodeEnd(Int32 index, Boolean close) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlDocument.cs:line 1805
   at HtmlAgilityPack.HtmlDocument.Parse() in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlDocument.cs:line 1492
   at HtmlAgilityPack.HtmlDocument.Load(TextReader reader) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlDocument.cs:line 769
   at HtmlAgilityPack.HtmlDocument.Load(Stream stream, Boolean detectEncodingFromByteOrderMarks) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlDocument.cs:line 597
   at HtmlAgilityPack.HtmlWeb.Get(Uri uri, String method, String path, HtmlDocument doc, IWebProxy proxy, ICredentials creds) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1515
   at HtmlAgilityPack.HtmlWeb.LoadUrl(Uri uri, String method, WebProxy proxy, NetworkCredential creds) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1563
   at HtmlAgilityPack.HtmlWeb.Load(String url, String method) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1152
   at HtmlAgilityPack.HtmlWeb.Load(String url) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1107
   at Conduit.CPServices.Logic.HtmlContentMonitor.HtmlAgilityPackHtmlProvider.Init(Url rootUrl) in D:\Conduit\RnD\Server\Services\CP\CPServices\Logic\HtmlContentMonitor\Conduit.CPServices.Logic.HtmlContentMonitor\HtmlAgilityPackHtmlProvider.cs:line 22
   at Conduit.CPServices.Logic.HtmlContentMonitor.HtmlContentManager.FetchRootAndExternlContentAsByteArray(Url rootUrl) in D:\Conduit\RnD\Server\Services\CP\CPServices\Logic\HtmlContentMonitor\Conduit.CPServices.Logic.HtmlContentMonitor\HtmlContentManager.cs:line 112

Answer 1

This is a bug in HtmlAgilityPack which could be caught e.g. if document encoding set through the <META> tag is invalid (e.g. <META http-equiv="Content-Type" content="text/html; charset=8859-9">). As Simon Mourier said, this is a bug introduced in 1.4.0.0.

See the answer to the similar error: The HAL internals tries to get an appropriate encoding for this string by using something like Encoding.GetEncoding("8859-9") and this throws an error.

To avoid this set the encoding manually, e.g.:

web.Load(rootUrl.Value, Encoding.GetEncoding("iso-8859-9"));