Question

I have a python script that I am writing for a class assignment which calculates the top 10 most frequent words in a text document and displays the words and their frequency. I was able to get this part of the script working just fine, but the assignment says a word is defined as 2 letters or more. I cannot seem to define a word as 2 letters or more for some reason, when I run the script, nothing happens.

# Most Frequent Words:
from string import punctuation
from collections import defaultdict

def sort_words(x, y):
    return cmp(x[1], y[1]) or cmp(y[0], x[0])

number = 10
words = {}

words_gen = (word.strip(punctuation).lower() for line in open("charactermask.txt")
                                             for word in line.split())
words = defaultdict(int)
for word in words_gen:
    words[word] +=1

letters = len(word)

while letters >= 2:
    top_words = sorted(words.iteritems(),
                        key=lambda(word, count): (-count, word))[:number] 

for word, frequency in top_words:
    print "%s: %d" % (word, frequency)

Answer 1

I would refactor your code and use a collections.Counter object:

import collections
import string

with open("charactermask.txt") as f:
  words = [x.strip(string.punctuation).lower() for x in f.read().split()]

counter = collections.defaultdict(int):
for word in words:
  if len(word) >= 2:
    counter[word] += 1