Home/ Questions/Q 113247
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-11T02:40:20+00:00

I’m working on a simple HTML scraper for Hulu in python 2.6 and am

  • 0

I’m working on a simple HTML scraper for Hulu in python 2.6 and am having problems with logging on to my account. Here’s my code so far:

import urllib import urllib2 from cookielib import CookieJar #make a cookie and redirect handlers cookies = CookieJar()  cookie_handler= urllib2.HTTPCookieProcessor(cookies) redirect_handler= urllib2.HTTPRedirectHandler()  opener = urllib2.build_opener(redirect_handler,cookie_handler)#make opener w/ handlers  #build the url login_info = {'username':USER,'password':PASS}#USER and PASS are defined data = urllib.urlencode(login_info)  req = urllib2.Request('http://www.hulu.com/account/authenticate',data)#make the request test = opener.open(req) #open the page print test.read() #print html results

The code compiles and runs, but all that prints is:

Login.onError('Please \074a href=\'/support/login_faq#cant_login\'\076enable cookies\074/a\076 and try again.');

I assume there is some error in how I’m handling cookies, but just can’t seem to spot it. I’ve heard Mechanize is a very useful module for this type of program, but as this seems to be the only speed bump left, I was hoping to find my bug.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. The error message you are getting back could be misleading. For example the server might be looking at user-agent and seeing that say it’s not one of the supported browsers, or looking at HTTP_REFERER expecting it to be coming from hulu domain. My point is there are two many variables coming in the request to keep guessing them one by one

    I recommend using an http analyzer tool, e.g. Charles or the one in Firebug to figure out what (header fields, cookies, parameters) the client sends to server when you doing hulu login via a browser. This will give you the exact request that you need to construct in your python code.

    • 0