Question

I’m using mechanize to visit the same web-site from different proxies. The website has a log-in page. I have 5 proxies and 5 different log-ins, one to use for each proxy.

If I just run my script with one proxy and one log-in, each one works fine. However, if I run two or more proxies/log-ins at once, then I start getting errors (from the website) like “either your session has timed out or cookies are not enabled.” This happens whether I’m running the 5 instances from the same script (same process), or from different scripts (different processes).

What would cause this to work individually but not all at once?

Answer 1

That is because mechanize automatically creates a shared “cookie jar” by default. For more advanced cookie handling options, you will have to create your own cookie jar for each of the script sessions.

I had to use a custom cookie jar in a past project, in order to move cookies from one session to another. The end result is the same, each instance of your script would have it’s own unique file to store it’s cookies, so it’s on you to manage the cookie files and know that they don’t get confused.

>>>> import mechanize

>>>> cj1 = mechanize.CookieJar()
>>>> cj2 = mechanize.CookieJar()
>>>> mech1 = mechanize.OpenerFactory().build_opener(mechanize.HTTPCookieProcessor(cj1))
>>>> mech2 = mechanize.OpenerFactory().build_opener(mechanize.HTTPCookieProcessor(cj2))

>>>> request = mechanize.Request('http://example.com') # testing shows they can share a request

>>>> response1 = mech1.open(request)
>>>> response2 = mech2.open(request)

>>>> print cj1
<mechanize._clientcookie.CookieJar[<Cookie JSESSIONID=54FBB2BE99E4CFDA8F8386F52FCF59C3>]>
>>>> print cj2
<mechanize._clientcookie.CookieJar[<Cookie JSESSIONID=350C0D544CDAD344A1272DA8D7B016B0>]>

In this example I’ve tested, you can see the two mechanize objects, each with it’s own independent cookie jar.