I am scraping a certain ASP.NET-enabled search page from a Python script and running into trouble getting the data I want from postbacks.
So I want to verify if I’ve got ASP.NET related bits correctly before ruling it out.
So far I just take all hidden form variables (e.g. __VIEWSTATE) and send them with other POST request parameters. Is this correct or they should change in some way? POST response contains updated hidden variables, which I assume should become parameters of the next postback, however when I browse the same page in Firefox, not all hidden form fields posted back are the same (e.g. __EVENTTARGET and __EVENTARGUMENT).
Do I need any other special handling here or this is all there is to it?
That’s pretty much all there is to it. The main thing is maintaining __VIEWSTATE from one post to another; which it sounds like you’ve covered.