Question

I have a list of words, in Python I need to loop through each word and check if the word is on a website.

Currently, this is a snippet of what I have (relating to this problem):

words = ['word', 'word1', 'word2']
site = urllib.request.urlopen(link)
for word in words:
    if word in site:
       print(word)
    else:
       print(word, "not found")

I have a list of words, I open the site, and I loop through each word checking for the word in the site. Note that I am using a website with all those words found on it (I set it up myself and I can verify it works) and the link is the url of the website.

The problem is, I always go to “word not found”, and it never seems to find the words on the website.

What’s wrong with the code? It seems to be a semantics error, because the syntax works fine, and there are no exceptions thrown (although in my final I do have exception handling, but it will still report if exceptions are thrown anyways).

Answer 1

The urlopen() function returns a “file-like object”. In order to read the data, you must call read():

site = urllib.request.urlopen(link).read()

There are other ways to read the data too, but this is a simple way to load the whole page data into memory for quick searching.

The reason your code worked as written is because a file-like object is also iterable, which means it can be used with the in operator. But it wasn’t doing what you wanted.