Question

I have a log file containing search queries entered into my site’s search engine. I’d like to “group” related search queries together for a report. I’m using Python for most of my webapp – so the solution can either be Python based or I can load the strings into Postgres if it is easier to do this with SQL.

Example data:

dog food
good dog trainer
cat food
veterinarian

Groups should include:

cat:
cat food

dog:

dog food
good dog trainer

food:

dog food
cat food

etc…

Ideas? Some sort of “indexing algorithm” perhaps?

Answer 1

f = open('data.txt', 'r')
raw = f.readlines()

#generate set of all possible groupings
groups = set()
for lines in raw:
    data = lines.strip().split()
    for items in data:
        groups.add(items)

#parse input into groups
for group in groups:
    print "Group \'%s\':" % group
    for line in raw:
        if line.find(group) is not -1:
            print line.strip()
    print

#consider storing into a dictionary instead of just printing

This could be heavily optimized, but this will print the following result, assuming you place the raw data in an external text file:

Group 'trainer':
good dog trainer

Group 'good':
good dog trainer

Group 'food':
dog food
cat food

Group 'dog':
dog food
good dog trainer

Group 'cat':
cat food

Group 'veterinarian':
veterinarian