Question

I have the script below, which modifies href attributes in an HTML file (in the future, it will be a list of HTML files in a directory). Using BeautifulSoup I managed to access the tag values and modify them like I want, but I don’t know how to save back the changes made to the file.

import os
import re
from bs4 import BeautifulSoup


htmlDoc = open('adding_computer_c.html',"r+")
soup = BeautifulSoup(htmlDoc)

replacements= [ ('_', '-'), ('../tasks/', prefixUrl), ('../concepts/', prefixUrl) ]

for link in soup.findAll('a', attrs={'href': re.compile("../")}):


    newlink=str(link)

    for k, v in replacements:

        newlink = newlink.replace(k, v)

    extrachars=newlink[newlink.find("."):newlink.find(">")]
    newlink=newlink.replace(extrachars,'')


    link=newlink
    print(link)
    ##How do I save the link I have modified back to the HTML file?

print(soup)##prints the original html tree

htmlDoc.close()

Answer 1

newlink = link['href']
# .. make replacements
link['href'] = newlink # store it back

Now print(soup.prettify()) will show changed links. To save the changes to a file:

htmlDoc.close()

html = soup.prettify("utf-8")
with open("output.html", "wb") as file:
    file.write(html)

To preserve original character encoding of the document, you could use soup.original_encoding instead of “utf-8”. See Encodings.