Question

I have two lists, let’s say:

a = [1,2,3]
b = [1,2,3,1,2,3]

I would like to remove 1, 2 and 3 from list b, but not all occurrences. The resulting list should have:

b = [1,2,3]

I currently have:

for element in a:
    try:
        b.remove(element)
    except ValueError:
        pass

However, this has poor performance when a and b get very large. Is there a more efficient way of getting the same results?

EDIT

To clarify ‘not all occurrences’, I mean I do not wish to remove both ‘1’s from b, as there was only one ‘1’ in a.

Answer 1

I would do something like this:

from collections import defaultdict

a = [1, 2, 3]
b = [1, 2, 3, 1, 2, 3]

# Build up the count of occurrences in b
d = defaultdict(int)
for bb in b:
    d[bb] += 1

# Remove one for each occurrence in a
for aa in a:
    d[aa] -= 1

# Create a list for all elements that still have a count of one or more
result = []
for k, v in d.iteritems():
    if v > 0:
        result += [k] * v

Or, if you are willing to be slightly more obscure:

from operator import iadd

result = reduce(iadd, [[k] * v for k, v in d.iteritems() if v > 0], [])

defaultdict generates a count of the occurrences of each key. Once it has been built up from b, it is decremented for each occurrence of a key in a. Then we print out the elements that are still left over, allowing them to occur multiple times.

defaultdict works with python 2.6 and up. If you are using a later python (2.7 and up, I believe), you can look into collections.Counter.

---

Later: you can also generalize this and create subtractions of counter-style defaultdicts:

from collections import defaultdict
from operator import iadd

a = [1, 2, 3, 4, 5, 6]
b = [1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3]

def build_dd(lst):
    d = defaultdict(int)
    for item in lst:
        d[item] += 1
    return d

def subtract_dd(left, right):
    return {k: left[k] - v for k, v in right.iteritems()}

db = build_dd(b)
da = build_dd(a)
result = reduce(iadd,
                [[k] * v for k, v in subtract_dd(db, da).iteritems() if v > 0],
                [])

print result

But the reduce expression is pretty obscure now.

---

Later still: in python 2.7 and later, using collections.Counter, it looks like this:

from collections import Counter

base = [1, 2, 3]
missing = [4, 5, 6]
extra = [7, 8, 9]
a = base + missing
b = base * 4 + extra

result = Counter(b) - Counter(a)
print result
assert result == dict([(k, 3) for k in base] + [(k, 1) for k in extra])