Question

From the following command how can i eliminate all the lines that occur before

 Owner     RepoName             CreatedDate

EDIT Command:

find /opt/site/ -name '.log.txt' | xargs cat | awk '{$NF=""; print $0}' | sed '1i Owner RepoName CreatedDate' | column -t

The output is

find: Filesystem loop detected; `/nfs/.snapshot/nightly.4' has the same device number and inode as a directory which is 2 levels higher in the filesystem hierarchy.
find: Filesystem loop detected; `/nfs/.snapshot/nightly.5' has the same device number and inode as a directory which is 2 levels higher in the filesystem hierarchy.
find: Filesystem loop detected; `/nfs/.snapshot/nightly.6' has the same device number and inode as a directory which is 2 levels higher in the filesystem hierarchy.
Owner     RepoName             CreatedDate
val        abc                  Fri          Mar  16  17:01:07  PDT
p1         repo_pc              Wed          Mar  21  11:34:42  PDT
New        fm                   Mon          Mar  19  00:15:51  PD 

Required output is only:

Owner     RepoName             CreatedDate
val        abc                  Fri          Mar  16  17:01:07  PDT
p1         repo_pc              Wed          Mar  21  11:34:42  PDT
New        fm                   Mon          Mar  19  00:15:51  PD 

Answer 1

Those find errors will be on stderr, so bypass your chain entirely, you’ll want to redirect the errors with 2>/dev/null, although that will prevent you seeing any other errors in the find command.

find /opt/site/ -name '.log.txt' 2>/dev/null | xargs cat | awk '{$NF=""; print $0}' | xargs sed "/Filesystem/d" | sed '1i Owner RepoName CreatedDate' | column -t

In general with a complicated command like this, you should break it down when you have errors so that you can work out where the problem is coming from.

Let’s split up this command to see what it’s doing:

find /opt/site/ -name '.log.txt' 2>/dev/null – find all the files under /opt/site/ named .log.txt

xargs cat – get all their contents, one after the other

awk '{$NF=""; print $0}' – delete the last column

xargs sed "/Filesystem/d" – Treat each entry as a file and delete any lines containing Filesystem from the contents of those files.

sed '1i Owner RepoName CreatedDate' – Insert Owner RepoName CreatedDate on the first line

column -t – Convert the given data into a table

I’d suggest building up the command, and checking the output is correct at each stage.

Several things are surprising about your command:

1. The find one looks for files that are exactly .log.txt rather than an extension.
2. The second xargs call – converting the contents of the .log.txt files into filenames.