Question

i have a big domain lists file for a proxy filter. In another file I have some exceptions i would like to remove from filter file all rows of excpetions file. Is it possibile with some “sed” operation?

Thanks.

Answer 1

You can generally use grep with the -v and -f options for this. In fact, you probably want to use fgrep or the -F flag as well to ensure the string are considered as fixed string rather than regexes. Without that, for example, the first line of the infile file below will be removed despite not actually matching the fixed string.

-v reverses the sense that that matching lines are thrown away rather than kept, and -f will get the patterns from a file rather than the command line.

For example:

pax> cat infile
http://wwwxdodgy.com/rest-of-url
http://www.dodgy.com/rest-of-url
ftp://this/one/is/good
https://www.bad.org/rest-of-url

pax> cat exceptions 
http://www.dodgy.com
https://www.bad.org

pax> fgrep -v -f exceptions infile 
ftp://this/one/is/good