I’m trying to scrape a web app that uses a few redirects and logouts/logins in between requests. I believe I’ll have to set AllowAutoRedirect to false so I can capture the redirect requests and manually redirect while watching for new cookies. My only experience with cookies is to just set the container and forget about it… do I have to parse the response headers to decide actions to take with cookies? Can someone lay out a general approach?
Update
It turns out that Chris was right. The redirects and cookies were working just fine… the application I was hitting did not like that I was not setting all the right headers (content type, user agent). After adding those in, I’m getting the response I expect.
You got the
CookieContainerpart right which most people miss so kudos to you!AllowAutoRedirectshould pick up cookies as redirects happen. Is there a reason that you need to manually process things?