Question

I have this HTML structure:

<div>
    <table>
       <tbody>
          <tr>
            <td>stuff</td>
          </tr>
          <tr>
            <td>
              <div>The content I want</div> 
           </td>
           </tr>
        </tbody>
      </table>
</div>

How do I get “the content I want” and delete all html tags?

Thanks

Answer 1

Use BeautifulSoup, e.g.

>>> from BeautifulSoup import BeautifulSoup as bs
>>> text = """<div>
...     <table>
...        <tbody>
...           <tr>
...             <td>stuff</td>
...           </tr>
...           <tr>
...             <td>
...               <div>The content I want</div> 
...            </td>
...            </tr>
...         </tbody>
...       </table>
... </div>"""
>>> a = bs(text)

Since all the <tr> tags have some content and you need the data from the second row, you cant just use a.text, but need to do something a bit more complex:

>>> a.table.tbody.findAll("tr")[1].div.text
u'The content I want'

Or, if there really is only one <div> tag on the table rows (<tr>), you can also just traverse the tags e.g:

>>> a.table.tbody.div.text
u'The content I want'

Or you can use the html parser from lxml module as follows:

>>> from lxml import html
>>> t = html.fromstring(text)
>>> t.xpath("table/tbody/tr[2]/td/div")[0].text
'The content I want'