Question

Given two strings, I’d like to be able to — in Python — be able to determine which words have been added and which words have been removed between the two. I’ve seen difflib, but apparently it cannot do it.

For example: given ‘hello my name is’ and ‘hello my guys is’, it would return [‘guys’] as added words, and [‘name’] as removed words. Thanks a lot.

EDIT: Probably the example I gave wasn’t the best. It should also work without spaces between the current text and the new text. Maybe using difflib to get all the new sections, and then split with regexp “\b”. I’ll give it a try.

Answer 1

This isn’t particularly pretty but seems to work for most cases I can think of. I’m sure this can be tidied up a lot too and should be easy to make case insensitive.

def freqs(list):
    words = {}
    for word in list:
        words[word] = words.get(word, 0) + 1
    return words

def added_and_removed(a, b):
    af = freqs(a.split())
    bf = freqs(b.split())

    removed = []
    added = []

    for key in af:
        num = bf.get(key)
        if num == None:
            if af[key] > 1:
                words = [key]*af[key]
                removed.extend(words)
            else:
                removed.append(key)

    for key in bf:
        num = af.get(key)
        if num == None:
            added.append(key)
        elif num > 1:
            words = [key]*(num-1)
            removed.extend(words)

    return added, removed

a = 'hello hello hello my name is Dave dave bar foo'
b = 'hello my guys is test easy rob dave beef foo'     

added, removed =  added_and_removed(a, b)
print added
print removed

gives

['beef', 'rob', 'easy', 'test', 'guys']
['bar', 'name', 'Dave', 'hello', 'hello']