Question

I am attempting to view the source of http://simpledesktops.com/browse/desktops/2012/may/17/where-the-wild-things-are/ using the code:

String URL = "http://simpledesktops.com/browse/desktops/2012/may/17/where-the-wild-things-are/";

WebClient webClient = new WebClient();

webClient.Headers.Add("user-agent", "Mozilla/5.0 (Windows; Windows NT 5.1; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4");
webClient.Encoding = Encoding.GetEncoding("Windows-1255");

string download = webClient.DownloadString(URL);

webClient.Dispose();

Console.WriteLine(download);

When I run this, the console returns a bunch of nonsense that looks like it’s been decoded incorrectly.

I’ve also attempted adding headers with no avail:

webClient.Headers.Add("user-agent", "Mozilla/5.0 (Windows; Windows NT 5.1; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4");    
webClient.Headers.Add("Accept-Encoding", "gzip,deflate");

Other websites all returned the proper html source. I can also view the page’s source through Chrome. What’s going on here?

Answer 1

Response of that URL is gzipped, you should decompress it or set empty Accept-Encoding header, you don’t need that user-agent field.

  String URL = "http://simpledesktops.com/browse/desktops/2012/may/17/where-the-wild-things-are/";    
  WebClient webClient = new WebClient();    
  webClient.Headers.Add("Accept-Encoding", "");
  string download = webClient.DownloadString(URL);