Question

I have a script which I pass a list of files to exclude from a spider function.

The problem is I have a list of files I want to include in the spider. Rather than rewrite the function to also allow passing in a list of files I want to include I would like to pull a list of all files in the directory and then remove the list of files I want to include from it.

So my thought was to do a find of the directory and pipe in the list of files I want to include to prune out of the list so I have a list of files to exclude.

This sounds confusing, and there is probably a better way to do it, but this is my first idea.

Answer 1

Maybe what you need is to sort the list of all files into file.1, sort the list of files to be excluded into file.2, and then use comm -23 file.1 file.2 to generate the files that can be allowed out.