Question

HTML structure I am working on is:

<tr showhide="show" style="display: table-row; ">
            <td class="monthlyCost">£18<span>.50</span></td>
                      <td class="phoneCost">£299.99</td>
                      <td><span class="lowLight">24 Months</span></td>
            <td>200</td>
            <td>Unlimited</td>
            <td class="dataAllowance">100MB</td>
            <td class="extras">

I am using following Python code to extract information:

minutes       = tariff_link_soup.findAll('td', {"class": False})
texts         = tariff_link_soup.findAll('td', {"class": False})

Problem: I want information mentioned in these two td (<td>200</td> & <td>Unlimited</td>) tags in 2 different columns minutes & texts. But my method which defines class as false, finds 3 arguments 24 Months , 200 & Unlimitedcontinually.

I understand the problem but not able to devise a solution for it. There are 2 parts to it, first I don’t want data in lowlight class , second I want data in next two td tags to be in 2 different columns, I can the separate the two in 2 columns even if I can get them together like 200 Unlimited

Please help me in resolving this issue.

Answer 1

If you’re using BeautifulSoup 3, the correct call would be:

.findAll('td', {'class': None})

That gives me:

>>> results = s.findAll('td', {'class': None})
>>> results
[<td><span class="lowLight">24 Months</span></td>, <td>200</td>, <td>Unlimited</td>]

As you’re calling this twice, you will get the same data in both minutes and texts. I assume your table’s columns are not going to change, so you’d have to use indexing:

>>> minutes = results[1]
>>> texts = results[2]
>>> minutes, texts
(<td>200</td>, <td>Unlimited</td>)