Question

I am trying to do three simple steps efficiently in Python.

I have a list of lists (of strings). Let us call it L.

1. I want to flatten the list of lists to a single list LL. (I know how to do this efficiently)

2. Construct the set of words with frequency 1 from the list LL of step 1. Let us call this set S. (I also know how to do this
   efficiently)

3. Remove all the words from the list of lists L which occur in S.

If you can suggest an efficient way of doing step 3, that will be a great help.

Answer 1

use a simple list comprehension for the 3rd step:

>>> from collections import Counter
>>> from itertools import chain
>>> L=[['a','b'],['foo','bar'],['spam','eggs'],['b','c'],['spam','bar']]
>>> S=Counter(chain(*L))
>>> S
Counter({'b': 2, 'bar': 2, 'spam': 2, 'a': 1, 'c': 1, 'eggs': 1, 'foo': 1})

>>> [[y for y in x if S[y]!=1] for x in L]
[['b'], ['bar'], ['spam'], ['b'], ['spam', 'bar']]

if you’ve a set R:

>>> L=[['a','b'],['foo','bar'],['spam','eggs'],['b','c'],['spam','bar']]
>>> R={'a','foo'}
>>> [[y for y in x if y not in R] for x in L]
[['b'], ['bar'], ['spam', 'eggs'], ['b', 'c'], ['spam', 'bar']]