Question

I am trying to subtract List_1 (50k lines) from List_2 (100k lines) , when an item in List_1 is an exact match for an item in List_2. I am using grep, specifically:

grep -v -f List_1.csv List_2.csv > Magic_List.csv

I know this is not the most efficient way to do this, but what is? sed? awk? comm? SQL? How might I accomplish this in the most efficient way possible?

Answer 1

This is one of the most efficient ways IMHO, you need to add -F though:

grep -Fvf List_1.csv List_2.csv > Magic_List.csv