Question

I am trying to download a file over HTTPS and I just keep running into a brick wall with correctly setting Cookies and Headers.

Does anyone have/know of any code that I can review for doing this correctly ? i.e. download a file over https and set cookies/headers ?

Thanks!

Answer 1

I did this the other day, in summary you need to create a HttpWebRequest and HttpWepResponse to submit/receive data. Since you need to maintain cookies across multiple requests, you need to create a cookie container to hold your cookies. You can set header properties on request/response if needed as well….

Basic Concept:

Using System.Net;

// Create Cookie Container (Place to store cookies during multiple requests)

CookieContainer cookies = new CookieContainer();

// Request Page
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://www.amazon.com");
req.CookieContainer = cookies;

// Response Output (Could be page, PDF, csv, etc...)

HttpWebResponse resp= (HttpWebResponse)req.GetResponse();

// Add Response Cookies to Cookie Container
// I only had to do this for the first "login" request

cookies.Add(resp.Cookies);

The key to figuring this out is capturing the traffic for real request. I did this using Fiddler and over the course of a few captures (almost 10), I figured out what I need to do to reproduce the login to a site where I needed to run some reports based on different selection critera (date range, parts, etc..) and download the results into CSV files. It’s working perfect, but Fiddler was the key to figuring it out.

http://www.fiddler2.com/fiddler2/

Good Luck.

Zach