Question

Basically, I’m trying to grab an EXE from CNet’s Download.com

So i created web parser and so far all is going well.

Here is a sample link pulled directly from their site:

http://dw.com.com/redir?edId=3&siteId=4&oId=3001-20_4-10308491&ontId=20_4&spi=e6323e8d83a8b4374d43d519f1bd6757&lop=txt&tag=idl2&pid=10566981&mfgId=6250549&merId=6250549&pguid=PlvcGQoPjAEAAH5rQL0AAABv&destUrl=ftp%3A%2F%2F202.190.201.108%2Fpub%2Fryl2%2Fclient%2Finstaller-ryl2_v1673.exe

Here is the problem: When you attempt to download, it begins with HTTP, then redirects to an FTP site. I have tried .NET’s WebClient and HttpWebRequest Objects, and it looks like Neither can support Redirects.

This Code Fails at GetResponse();

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://dw.com.com/redir");
WebResponse response = req.GetResponse();

Now, I also tried this:

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://dw.com.com/redir");
req.AllowAutoRedirect = false;
WebResponse response = req.GetResponse();
string s = new StreamReader(response.GetResponseStream()).ReadToEnd();

And it does not throw the error anymore, however variable s turns out to be an empty string.

I’m at a loss! Can anyone help out?

Answer 1

You can get the value of the “Location” header from the response.headers, and then create a new FtpWebRequest to download that resource.