Question

I want to let the user choose and open multiple texts and perform a search for exact matches in the texts.
I want the encoding to be unicode.

If I search for “cat” I want it to find “cat”, “cat,”, “.cat” but not “catalogue”.

I don’t know how to let the user search for two words (“cat” OR “dog”) in all of the texts at the same time??????
Maybe I can use RE?

So far I have just made it possible for the user to insert the path to the directory containing the text files to search in. Now I want to let the user (raw_input) search for two words in all of the texts, and then print and save the results (e.g. “search_word_1” and “search_word_2” found in document1.txt, “search_word_2” found in document4.txt) in a separate document (search_words).

import re, os


path = raw_input("insert path to directory :")
ex_library = os.listdir(path)
search_words = open("sword.txt", "w") # File or maybe list to put in the results
thelist = []

for texts in ex_library:
    f = os.path.join(path, texts)
    text = open(f, "r")
    textname = os.path.basename(texts)
    print textname
    for line in text.read():

    text.close()

Answer 1

Regular expressions are appropriate tool in this case.

I want it to find “cat”, “cat,”, “.cat” but not “catalogue”.

Pattern: r'\bcat\b'

\b matches at a word boundary.

how to let the user search for two words (“cat” OR “dog”) in all of the texts at the same time

Pattern: r'\bcat\b|\bdog\b'

To print "filename: <words that are found in it>":

#!/usr/bin/env python
import os
import re
import sys

def fgrep(words, filenames, encoding='utf-8', case_insensitive=False):
    findwords = re.compile("|".join(r"\b%s\b" % re.escape(w) for w in words),
                           flags=re.I if case_insensitive else 0).findall
    for name in filenames:
        with open(name, 'rb') as file:
             text = file.read().decode(encoding)
             found_words = set(findwords(text))
             yield name, found_words

def main():
    words = [w.decode(sys.stdin.encoding) for w in sys.argv[1].split(",")]
    filenames = sys.argv[2:] # the rest is filenames
    for filename, found_words in fgrep(words, filenames):
        print "%s: %s" % (os.path.basename(filename), ",".join(found_words))

main()

Example:

$ python findwords.py 'cat,dog' /path/to/*.txt

Alternative solutions

To avoid reading the whole file in memory:

import codecs

...
with codecs.open(name, encoding=encoding) as file:
    found_words = set(w for line in file for w in findwords(line))

You could also print found words in the context they are found e.g., print lines with highlighted words:

from colorama import init  # pip install colorama
init(strip=not sys.stdout.isatty())  # strip colors if stdout is redirected
from termcolor import colored  # pip install termcolor

highlight = lambda s: colored(s, on_color='on_red', attrs=['bold', 'reverse'])

...
regex = re.compile("|".join(r"\b%s\b" % re.escape(w) for w in words),
                   flags=re.I if case_insensitive else 0)

for line in file:
    if regex.search(line): # line contains words
       line = regex.sub(lambda m: highlight(m.group()), line)
       yield line