Question

I have to parse (scrap) a webpage (html) and get some information, store it in a csv.

I have to do this using both Regex and BeautifulSoup, and I have absolute no prior experience with Python.

I’ll really appreciate if anyone can guide me accordingly for the code. I have managed to install python and BeautifulSoup.

Answer 1

For python 3 :
Read a web page with the following commands:

import urllib.request

your_url = "http://stackoverflow.com/"   # for example

opener = urllib.request.FancyURLopener({})
f = opener.open( your_url )
vr = f.read()
vr = vr.decode()
print( vr )

And match some regexp, for example, with:

import re

re_flags = ( re.MULTILINE | re.IGNORECASE | re.UNICODE )
m = re.search( r"<div>\s*(\d*)\s*</div>" , vr , re_flags )
if m :
    print( m.groups()[0] )

If it find the regexp the matched string is written in m.groups()[0]

And clearly you must use your personal regexp in re.search

Visit:
http://docs.python.org/library/re.html