Question

I’m attempting to programmatically log in to the Yahoo developer API. I’ve hit a stumbling block where I can’t access all of the “Set-Cookie” headers of a HttpWebResponse.

Fiddler shows me the following cookies in the header of the response:

Set-Cookie: B=733jjvp7f5g8f&b=4&d=1pFN8bVpYFYaPUme88.fc6ZzTSI-&s=kc&i=.1p3Ei3yvwqZjo0gcg7D; expires=Sun, 22-Dec-2013 05:33:04 GMT; path=/; domain=.yahoo.com
Set-Cookie: F=a=GYsABKAMvTZoTcNAPKUXrclX_Hb77EA7I_62nONz8QeEwNevHwqJ_NyizED88uhv9aMx.9o-&b=3tN5; expires=Sun, 22-Dec-2013 05:33:04 GMT; path=/; domain=.yahoo.com
Set-Cookie: Y=v=1&n=0v251rt3ifppb&l=0kii84if0h70ma/o&p=m2fvvau012000000&iz=1111&r=if&lg=en-AU&intl=au&np=1; path=/; domain=.yahoo.com
Set-Cookie: PH=fn=jW23i4lnq1UpiP.lsuU-&l=en-AU; expires=Sun, 22-Dec-2013 05:33:04 GMT; path=/; domain=.yahoo.com
Set-Cookie: T=z=QEs8OBQYTBPBEZq31nTCqv1MzNPBjUwTjcwMDZOTjY-&a=YAE&sk=DAAtoxgrYmWIMk&ks=EAA3Ha0H7qyCT8P3cI9NWJrIA--~E&d=c2wBTkRRNEFUSTNPVEEzTnpFNU9URS0BYQFZQUUBZwFCRFZQTkRSSjJQRVRDTEdFT0xCQ1hER0VVUQFvawFaVzAtAXRpcAF2MkNUVUEBenoBUUVzOE9CQTdF; path=/; domain=.yahoo.com
Set-Cookie: SSL=v=1&s=kTc532PQYAe1iT.23Q55E50ZdoOAdEK_fshc3g_YZ3SxszcbuHkmpJUAQ7RT67nDNA0nXyX68um90ZuS9RQztQ--&kv=0; path=/; domain=.yahoo.com; secure; httponly

However, I’m unable to access anything beyond the first instance of “Set-Cookie” via .NET:

// Make the web request:
var userAuthWebRequest = WebRequest.Create(uri) as HttpWebRequest;
var response = userAuthWebRequest.GetResponse() as HttpWebResponse;

// Dump the headers to debug:
Debug.WriteLine(string.Format("Set-Cookie: {0}", response.Headers.Get("Set-Cookie")));

My debug output returns:

Set-Cookie: B=733jjvp7f5g8f&b=3&s=b1; expires=Sun, 22-Dec-2013 05:33:03 GMT; path=/; domain=.yahoo.com

Interestingly, if I make a similar request to Headers.GetValues, it actually returns two instances of the “Set-Cookie” header, which appear to have been concatenated together above:

foreach (var headerName in response.Headers.AllKeys)
{
    foreach (var values in response.Headers.GetValues(headerName))
    {
        Debug.WriteLine("{0}: {1}", headerName, values);
    }
}

Output:

Set-Cookie: B=733jjvp7f5g8f&b=3&s=b1; expires=Sun
Set-Cookie: 22-Dec-2013 05:33:03 GMT; path=/; domain=.yahoo.com

I’ve seen a few other questions that confirm the raw headers aren’t available via the HttpWebResponse object, and I should investigate using a sockets solution. I’m about to check out FiddlerCore to see if I have any luck with that, but if anyone has any other pointers I’d love to see them.

Thanks!

Answer 1

You’ll typically want to use the ‘CookieCollection’ class to handle parsing and handling the set-cookie headers.

For help on how to implement it check out the example from MSDN (HttpWebRequest.CookieContainer):

using System.Net;
using System;
namespace Examples.System.Net.Cookies
{
    // This example is run at the command line.
    // Specify one argument: the name of the host to 
    // send the request to.
    // If the request is sucessful, the example displays the contents of the cookies
    // returned by the host.

    public class CookieExample
    {   
        public static void Main(string[] args)
        {   
            if (args == null || args.Length != 1)
            {
                Console.WriteLine("Specify the URL to receive the request.");
                Environment.Exit(1);
            }
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(args[0]);
            request.CookieContainer = new CookieContainer();

            HttpWebResponse response = (HttpWebResponse) request.GetResponse();



            // Print the properties of each cookie.
            foreach (Cookie cook in response.Cookies)
            {
                Console.WriteLine("Cookie:");
                Console.WriteLine("{0} = {1}", cook.Name, cook.Value);
                Console.WriteLine("Domain: {0}", cook.Domain);
                Console.WriteLine("Path: {0}", cook.Path);
                Console.WriteLine("Port: {0}", cook.Port);
                Console.WriteLine("Secure: {0}", cook.Secure);

                Console.WriteLine("When issued: {0}", cook.TimeStamp);
                Console.WriteLine("Expires: {0} (expired? {1})", 
                    cook.Expires, cook.Expired);
                Console.WriteLine("Don't save: {0}", cook.Discard);    
                Console.WriteLine("Comment: {0}", cook.Comment);
                Console.WriteLine("Uri for comments: {0}", cook.CommentUri);
                Console.WriteLine("Version: RFC {0}" , cook.Version == 1 ? "2109" : "2965");

                // Show the string representation of the cookie.
                Console.WriteLine ("String: {0}", cook.ToString());
            }
        }
    }
}