Question

Trying to get a bit more out of the command line:

I have a file:

CHR25   1192279 Y       Exonic  NA      exon    2       [1/1]   1192145 1192322
CHR25   1192279 Y       Exonic  NA      CDS     2       [1/1]   1192145 1192322  
CHR25   1739394 Y       Exonic  NA      CDS     2       [1/2]   1739294 1739456

I want to get unique lines based on column 2. I have done

cat file | cut -f 2 | uniq > grepfile

but am unsure of how to now get one match per line using grep -f as I previously have. Using -m1 or | head -n 1 only gives the first matching line. I want the first line to match each line in ‘grepfile’.

Look forward to answers=)

Answer 1

This awk command works by building an associative array of second-field values that have been seen already.

awk '!s[$2]++'

The associative array is called s. The expression looks up $2 (the second column of the input line) and applies a postincrement, which will return 0 the first time it’s called and nonzero thereafter. Then I invert it, yield 1 the first time each $2 is seen, and 0 every time afterward. And awk’s default action is to print if the selecting expression is true.

Just watch out when the line count is big enough to wrap around to 0 again.