Question

i’m trying to scrape a little page (with POST method) written in asp.net. But there is a problem, i can’t acces to the session ID specified by the server in the URL “…/(S(XXXXXXXXX))/…”.

This is my code:

import urllib
import urllib2

headers = {
    'HTTP_USER_AGENT': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13',
    'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml; q=0.9,*/*; q=0.8',
    'Content-Type': 'application/x-www-form-urlencoded'
}

url = 'http://mobile.trenitalia.com/W2/ViaggiaTrenoXHTML.aspx'
values = {'Text1':21324,
            '__EVENTVALIDATION':'/wEWAwLgwZLkDQLzlKGwCgK28MDdBaZpixsZK6pgmdSd5WzAhOKq22MJ',
            '__VIEWSTATE':'/wEPDwUINzkxNzIxNzFkZCx1ajW4bJv22/lexpR1EWM3e10A',
            'butt_Find':'Cerca'}

data = urllib.urlencode(values)
req = urllib2.Request(url)#, data, headers)
response = urllib2.urlopen(req)
the_page = response.read()
print the_page

how you can see the url is:
http://mobile.trenitalia.com/W2/ViaggiaTrenoXHTML.aspx

and there is no trace of the session, how can i access to the session given by the server?

Answer 1

On your response variable, you can call the geturl() method to get the resulting URL:

In [13]: response.geturl()
Out[13]: 'http://mobile.trenitalia.com/(S(saaizweqpa3q5yvnd4qjytii))/W2/ViaggiaTrenoXHTML.aspx'