Question

I am trying to process a large number of text files. These text files contain either of the following two consecutive lines:

“_atom_site_fract_z” followed by “#END”

or

“_atom_site_fract_z” followed by strings such as “C1 C 0.46450 0.18880 0.92540”

I want to use bash/sed to only keep the files that are of the later type (files that does NOT have “_atom_site_fract_z” followed by “#END”).

How do I achieve this?

NOTE: Two strings are separated by a NEWLINE. They are not separated by a space.

UPDATE: The name of files are stored in a text file, and I want to read the text file, line by line, to check if I should keep the file or not. I do not necessarily want to delete them, but want to save the files that are of later type in a separate folder within the directory.

UPDATE2: There are “other lines” besides these two lines. I want to search the file that has the particular combination of two lines. ALL files have both “_atom_site_frac_z” and “#END”, but they don’t appear immediately after one another. However, “_atom_site_frac_z” ALWAYS appear before “#END”.

Answer 1

You say you want to keep only files of the latter type. sed might be useful for dealing with lines, but for whole files you probably want grep with find.

find "$dir" -type f -exec grep -qF '_atom_site_fract_z#END' {} \; -print # get a list of the files to delete.
find "$dir" -type f -exec grep -qF '_atom_site_fract_z#END' {} \; -delete # actually delete them

Update

If your files are from a list in a newline-separated textfile then you can process them like this:

while read filename; do
    awk '!/#END/{
        checkNext=0;
    } /_atom_site_fract_z/{
        checkNext=1;
        next;
    } /#END/{
        if (checkNext) {
            print(FILENAME);
            exit(0);
        }
    }' "$filename"
done < list_of_files.txt