Question

I have tried to find a table with class “data” in a web page with this code.

import urllib2
from BeautifulSoup import BeautifulSoup

soup = BeautifulSoup(urllib2.urlopen('http://www.cbssports.com/nba/draft/mock-draft').read())

rows = soup.findAll("table.data")
print rows

However, I am getting none for rows even though I am sure that a table with class “data” exists on that page. What is the proper way to find an element with class “data” on a web page with BeautifulSoup?

Answer 1

If you want to pick up the rows, you’ll need the following

import urllib2
from BeautifuSoup import BeautifulSoup

soup = BeautifulSoup(urllib2.urlopen('http://www.cbssports.com/nba/draft/mock-draft').read())

# if there's only one table with class = data
table = soup.find('table', attrs = {'class' : 'data'})

# if there are multiple tables with class = data
table = soup.findAll('table', attrs = {'class' : 'data'})[n]
# suppose you need the n-th table of the list returned

rows = table.findAll('tr') # gives all the rows, you can set attrs to filter

Then you can also iterate through the columns:

for row in rows:
    cols = row.findAll('td')
    ...