Question

Is there a faster way to do this in python?

[f for f in list_1 if not f in list_2]

list_1 and list_2 both consist of about 120.000 strings. It takes about 4 minutes to generate the new list.

Answer 1

If you put list_2 into a set, it should make the containment checking a lot quicker:

s = set(list_2)
[f for f in list_1 if not f in s]

This is because x in list is an O(n) check, while x in set is constant-time.

Another way is to use set-difference:

list(set(list_1).difference(set(list_2)))

However, this probably won’t be faster than the first way – also, it’ll eliminate duplicates from list_1 which you may not want.