Question

I am writing a program to scrape a monster.com job search and print out all the job titles. When i run it prints nothing, so i added a list range (which is commented out now) to see if the list was populating. It is not. Please let me know what I did wrong.

Here is the code. I’m new to python but to me this should work.

Thanks in advance.

    #! /usr/bin/python
import re
import requests
from urllib import urlopen
from BeautifulSoup import BeautifulSoup

webpage = urlopen("http://jobsearch.monster.com/search/Engineer_5?q=Software&where=AZ&rad=20&sort=rv.di.dt").read()

listIterator = []
#listIterator[:] = range(1,50)

soup = BeautifulSoup(webpage)

titleSoup = soup.findAll("title")

for i in listIterator:
    print titleSoup[i]
    print "\n"

raw_input("Press enter to close: ")

Answer 1

from urllib import urlopen
from BeautifulSoup import BeautifulSoup

handle = urlopen("http://jobsearch.monster.com/search/Engineer_5?q=Software&where=AZ&rad=20&sort=rv.di.dt")
responce = handle.read()
soup = BeautifulSoup( responce )

job_urls = soup.findAll(name = 'a', attrs = { 'class': 'jobTitle fnt11_js' })
for job_url in job_urls:
    print job_url.text
    print