Question

I don’t understand this error. How do I get “content” to become writeable?

from bs4 import BeautifulSoup

soup = BeautifulSoup(open("http://www.asdf.fi/asdf.html"))

content = soup.find(id="content") 

with open("test.html", "a") as myfile:
    myfile.write(content)

Error:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
TypeError: expected a character buffer object

Answer 1

First, you cannot open a web page using open(). You’ll need to use the urllib library (actually I use the mechanize library, it’s easier to use).

Second, open() returns a file object, which cannot be passed to BeautifulSoup(). You’ll need to write something like

soup = BeautifulSoup(open(filename).read())

.read() reads the whole file and returns the character buffer, which can be used to call BeautifulSoup().