Question

I have this code:

private List<string> webCrawler(string url, int levels)
        {
            HtmlAgilityPack.HtmlDocument doc;
            HtmlWeb hw = new HtmlWeb(); 
            List<string> webSites;
            List<string> csFiles = new List<string>();

            csFiles.Add("temp string to know that something is happening in level = " + levels.ToString());
            csFiles.Add("current site name in this level is : "+url);

            doc = hw.Load(url);
            webSites = getLinks(doc);


            if (levels == 0)
            {
                return csFiles;
            }
            else
            {
                int actual_sites = 0;
                for (int i = 0; i < webSites.Count() && i< 20; i++)                 {
                    string t = webSites[i];
                                        if ( (t.StartsWith("http://")==true) || (t.StartsWith("https://")==true) )                     {
                        actual_sites++;
                        csFiles.AddRange(webCrawler(t, levels - 1));
                        Texts(richTextBox1, "Level Number " + levels + " " + t + Environment.NewLine, Color.Red);
                    }
                }

                return csFiles;
            }


        }

And getLinks() is:

private List<string> getLinks(HtmlAgilityPack.HtmlDocument document)
        {

            List<string> mainLinks = new List<string>();
            var linkNodes = document.DocumentNode.SelectNodes("//a[@href]");
            if (linkNodes != null)
            {
                foreach (HtmlNode link in linkNodes)
                {
                    var href = link.Attributes["href"].Value;
                    mainLinks.Add(href);
                }
            }
            return mainLinks;

        }

The problem is for example i crawl into google.com so after few times its getting to the site:

http://picasa.google.co.il/intl/iw/#utm_source=iw-all-more&utm_campaign=iw-pic&utm_medium=et

Then im getting the exception on the line:

doc = hw.Load(url);

The error is: The remote name could not be resolved: ‘picasa.google.co.il’

The exception is:

System.Net.WebException was unhandled
  Message=The remote name could not be resolved: 'picasa.google.co.il'
  Source=System
  StackTrace:
       at System.Net.HttpWebRequest.GetResponse()
       at HtmlAgilityPack.HtmlWeb.Get(Uri uri, String method, String path, HtmlDocument doc, IWebProxy proxy, ICredentials creds) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1446
       at HtmlAgilityPack.HtmlWeb.LoadUrl(Uri uri, String method, WebProxy proxy, NetworkCredential creds) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1563
       at HtmlAgilityPack.HtmlWeb.Load(String url, String method) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1152
       at HtmlAgilityPack.HtmlWeb.Load(String url) in C:\Source\htmlagilitypack\Trunk\HtmlAgilityPack\HtmlWeb.cs:line 1107
       at GatherLinks.Form1.webCrawler(String url, Int32 levels) in D:\C-Sharp\GatherLinks\GatherLinks\GatherLinks\Form1.cs:line 79
       at GatherLinks.Form1.webCrawler(String url, Int32 levels) in D:\C-Sharp\GatherLinks\GatherLinks\GatherLinks\Form1.cs:line 108
       at GatherLinks.Form1.webCrawler(String url, Int32 levels) in D:\C-Sharp\GatherLinks\GatherLinks\GatherLinks\Form1.cs:line 108
       at GatherLinks.Form1..ctor() in D:\C-Sharp\GatherLinks\GatherLinks\GatherLinks\Form1.cs:line 31
       at GatherLinks.Program.Main() in D:\C-Sharp\GatherLinks\GatherLinks\GatherLinks\Program.cs:line 18
       at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)
       at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
  InnerException: 

How can i repair/fix/resolve that ?

Thank you.

Answer 1

The exception is telling you that it can’t resolve picasa.google.co.il to an IP address. You probably just need to verify that the name is correct.

Open a command window and type:

ping picasa.google.co.il

You’ll find that your computer can’t talk to this server because there isn’t a DNS entry for it.