Question

I am writing an application that needs to download a webpage. however i use this code scheme to get html :

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(address);                
HttpWebResponse response = (HttpWebResponse)request.GetResponse();                

System.IO.Stream data = response.GetResponseStream();
System.IO.StreamReader reader = new System.IO.StreamReader(data, true);
string htmlCode = reader.ReadToEnd();

i have a problem with urls from this site : http://www.jamejamonline.ir/

When i try to get a page in this way, redirected to error page,
but all webbrowsers show correct html page.

how can i get around this problem?

Answer 1

specify an user agent like this

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(address);  
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1";                
HttpWebResponse response = (HttpWebResponse)request.GetResponse();                

System.IO.Stream data = response.GetResponseStream();
System.IO.StreamReader reader = new System.IO.StreamReader(data, true);
string htmlCode = reader.ReadToEnd();

I tried it and it worked!