Question

I need to do a seemingly simple thing in Python which turned out to be quite complex. What I need to do is:

1. Open an HTML file.
2. Match all instances of a specific HTML element, for example table.
3. For each instance, extract the element as a string, pass that string to an external command which will do some modifications, and finally replace the original element with a new string returned from the external command.

I can’t simply do a re.sub(), because in each case the replacement string is different and based on the original string.

Any suggestions?

Answer 1

Sounds like you want BeautifulSoup. Likely, you’d want to do something like:

from bs4 import BeautifulSoup
soup = BeautifulSoup(html_doc)

tables = soup.find_all( 'table' )
for table in tables:
    contents = str( table.contents )
    new_contents = transform( contents )
    table.replaceWith( new_contents )

Alternatively, you may be looking for something closer to soup.replace_with

EDIT: Updated to the eventual solution.