Question

I’m fairly new to this, and don’t have anyone else to ask. I’m attempting to access a webpage programmatically (C#) for some web-scraping software. I’ve figured out the process of using the HttpWebResponse and HttpWebRequest classes to perform a login through a web page form, but I have a website which has a two stage login procedure.

The first login occurs via a pop-up. No webpage is loaded in the background, just a pop-up dialog saying authentication is required and prompts for a username and password.

After getting past the first login, the second login is simply a web form which I should be able to handle myself from what I’ve already learned.

My question: How do I programmatically provide the login information for the pop-up authentication request?

EDIT

Just to provide some more information. Fiddler gives the Host as CONNECT when attempting to access the website.

EDIT

I attempted to use MrEye’s answer below, and though it seemed to get me a little further, still having hangups. I was gettin certificate issues, so after some Google work I found a workaround for it by adding the following code:

ServicePointManager.ServerCertificateValidationCallback +=
            delegate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors errors)
        {
            return true;
        };

Now I’m getting a System.Net.WebException with the value "The remove server returned an error: (401) Unauthorized." The exception’s Status is ProtocolError, though I haven’t been able to get any further. I know for certain that the username/password being provided are valid.

Answer 1

All you need to do is asign a NetworkCredential object to the Credentials property of the HttpWebRequest object you have created. For example:

myWebRequest.Credentials = new NetworkCredential("username","password");

Depending on the server you are connecting to you may also need:

myWebRequest.PreAuthenticate = true;