Question

I am trying to scrape data from a webpage that runs on Asp.Net Webforms. I had a look at this page for doing this. It looks like what i wan’t to do, but i’m not gonna get it to work out of the box, because the login page requires a cookie to be presented. The cookie is set on a page that sets the cookie, and then redirects the user to the login page. How should i modify the code on the link so that it first browses the redirect-page, saves the cookie and then passes the cookie to the login page?

Answer 1

Worked great =)

CookieContainer cookieJar = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("http://mydomain.com/Start.aspx?g=4");
request.CookieContainer = cookieJar;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

CookieContainer cookies = new CookieContainer();
HttpWebRequest request1 = (HttpWebRequest)WebRequest.Create("http://mydomain.com/Login.aspx");
request1.CookieContainer = cookieJar;
HttpWebResponse response1 = (HttpWebResponse)request1.GetResponse();
StreamReader reader = new StreamReader(response1.GetResponseStream());
string loginPage = reader.ReadToEnd();
reader.Close();