Question

$file is a csv (tab-delimited) with 59 cols and 64 rows. Column 1 is always a string and cols 2+ are always an integer (except once when the value is NULL).

cat ${file} | while read line 
    do awk -F'\t' '{ for (i=2; i<=NF; i++) print $1 "\t" $i "." }';
    done;

Outputs:

Excellent   .
Good    .
…

And yet switching $i to $2 works:

Excellent   29.
Good    7.
…

Why??

EDIT:

#lines 1 & 2 from data.csv (columns truncated for brevity):
Excellent   29  54  47  46  38  22  50
Good    7   14  27  24  26  36  20

#reform.sh
file=$1;
awk -F'\t' '{ for (i=2; i<=NF; i++) print $1 "\t" $i; }' ${file};

Still behaves/returns like the previous.

RESOLUTION:

The approved answer does provide the proper output from AWK. After restarting my Terminal application, the script performed as described. I was not able to determine the cause of the Terminal app issue.

Answer 1

I think you’re mistaken. The script as you have it doesn’t output anything intelligent ^(a) because, while you’re attempting to read each line into $line, you’re not actually giving them to awk.

You can get rid of the superfluous (and incorrect) loop with something like:

awk -F'\t' '{ for (i=2; i<=NF; i++) print $1 "\t" $i "." }' ${file}

as shown in the following transcript:

pax> echo 'A 1 2
...> B 3 4
...> C 5 6' >qq.in

pax> cat qq.in
A 1 2
B 3 4
C 5 6

pax> awk -F' ' '{ for (i=2; i<=NF; i++) print $1 " " $i "." }' qq.in
A 1.
A 2.
B 3.
B 4.
C 5.
C 6.

As you can see (although I’m using spaces rather than tabs), this gives you the output you desire.

---

In response to your assertion that it’s still not working, I’m afraid I have to beg to differ. The following transcript (with tabs) shows that it works as advertised.

pax> cat qq.in
Excellent   29      54      47      46      38      22      50
Good        7       14      27      24      26      36      20

pax> awk -F'\t' '{ for (i=2; i<=NF; i++) print $1 "\t" $i; }' qq.in
Excellent   29
Excellent   54
Excellent   47
Excellent   46
Excellent   38
Excellent   22
Excellent   50
Good        7
Good        14
Good        27
Good        24
Good        26
Good        36
Good        20

If it’s not actually working in your environment, that’s a different issue. You may have a buggy awk or any other number of reasons why it would fail.

For a start, figure out what version of awk and the operating system you’re using, such as with:

awk --version
uname -a

---

^(a): It does actually output something, but almost certainly not what you’d expect. Let’s actually look at what happens in reality. Consider the following transcript which is similar to your original:

pax> ( echo 1; echo 2; echo 3 ) | while read line ; do
...>     awk '{print "[" $0 "]"}'
...> done
[2]
[3]

Now that looks rather strange, it appears to be throwing away the first line.

The reason for this is the disconnect between the while and the awk. The while reads the first line from standard input and assigns it to $line, then executes the body of the do..done section.

That body is an awk with no input file, hence it takes its input from standard input!

That means it will “suck up” the rest of your standard input stream and process it.

Then, it will return to the while loop but, with no more data on standard input, it will finish. It’s perhaps better illustrated with:

pax> ( echo 1; echo 2; echo 3 ) | while read line ; do
...>     echo "read: $line"
...>     awk '{print "awk:  " $0}'
...> done
read: 1
awk:  2
awk:  3

If you actually connect the while and the awk with the echo "$line" | section below, you’ll see it works properly:

pax> ( echo 1; echo 2; echo 3 ) | while read line ; do
...>     echo "$line" | awk '{print "[" $0 "]"}'
...> done
[1]
[2]
[3]

Of course, it makes little sense to break apart your input into lines and send them to awk one at a time, when awk is perfectly capable of handling multiple lines one at a time on its own.

So the single-line awk command shown in the first code block of this answer is still a better way to do it.