Question

I normally use the for construct in ksh to quickly iterate over a list of files to perform some action on it. It doesn’t seem to work in this scenario:

The file info looks like: 
$ ls -l tmp.*  
rw------- 1 op general 375 Jul 25 04:09 tmp.zzyhsg4  
...  
so on. Basically a lot of tmp.* files.    

Now when I try  
$ ls -lS | grep 'Jul 25' | grep 'tmp.*' | cut -d' ' -f9 | more  
tmp.zzyhsg4  
..  

it will print only the file names as expected.
However when I try the below

$ for i in `ls -lS | grep 'Jul 25' | grep 'tmp.*' | cut -d' ' -f9`
>do  
>echo $i  
>done  

This does not print the name of all the files starting with tmp.* which were created on Jul 25 sorted by size. It prints the size column. Interestingly if I replace the f10 by f6 for the cut it will correctly print the month column. It starts to break after f9.

Any ideas ?

Answer 1

I can’t reproduce exactly what you describe, but I have some suggestions to write more reliable commands.

cut -d' ' separates fields by spaces. If you have two spaces in a row, there’s an empty field between them. So if you try with Aug 1 instead of Jul 25, the file name column is shifted by 1. And if you try with files that are more than 6 months old, the (5-character) time is replaced by a space followed by the 4-digit year. Also, depending on your version of ls there may be more than one space between some columns. Yet another issue is that some versions of ls don’t display the group column. And then some file names contain spaces. And some file names contain special characters that ls may display as ?. In summary, you can’t parse the output of ls -l by counting spaces, and you can’t even parse the output of ls -l by counting whitespace-delimited fields. Just don’t parse the output of ls.

The standard command for generating lists of names of existing files is find. Since you mention Linux, I’ll mention options that work with GNU find (the version you get on Linux) but not on other unixes.

Let’s start simple: list the files called tmp.* in the current directory.

find . -name 'tmp.*'

We want only the files created on July 25, that’s 7 days ago.

find . -name 'tmp.*' -daystart -mtime 7

This is fragile since it won’t work tomorrow. The usual way to specify a precise date is to create files dated at the earliest and latest allowable times and tell find to only return files dated between these two.

touch -t 201007250000 .earliest
touch -t 201007260000 .latest
find . -name 'tmp.*' -newer .earliest \! -newer .latest
rm .earliest .latest

The find command explores subdirectories recursively. If you don’t want this:

find . -name 'tmp.*' -daystart -mindepth 1 -maxdepth 1 -mtime 7

If you want the files sorted by size:

find . -name 'tmp.*' -daystart -mtime 7 -printf '%s\t%p\n' | sort -n -k 1 | cut -f 2-

Finally, if you want to operate on the files, never use find in backticks, the way you used ls, because this will fail if the file names contain whitespace or some special characters, because the shell splits the output of `command` at whitespace and then does globbing on the resulting words. Instead, use the -exec option to find; the ; version executes mycommand once per file with {} replaced by the file name, whereas the + version usually invokes mycommand only once with {} replaced by the list of file names.

find . -name 'tmp.*' -daystart -mtime 7 -exec mycommand -myoption {} \;
find . -name 'tmp.*' -daystart -mtime 7 -exec mycommand -myoption {} +