Question

I would like to extract lines of a file where specific columns have a value <0.05.

For example if $2 or $4 or $6 has a value <0.05 then I want to send that line to a new file.
I do not want any lines that have a value >0.05 in any of these columns

cat File_1.txt 
S_003   P_003   S_006   P_006   S_008   P_008
74.9    0.006   59.6    0.061   72.2    0.002
96.2    0.003   89.4    0.001   106.9   0.000
105.8   0.003   72.6    0.003   86.7    0.002
45.8    0.726   38.5    0.981   43.9    0.800
50.7    0.305   47.8    0.314   46.6    0.615
49.9    0.366   50.4    0.165   48.2    0.392
42.5    0.920   43.7    0.698   40.3    0.970
46.3    0.684   42.9    0.760   47.7    0.438
192.4   0.001   312.8   0.001   274.3   0.001

I tried this using awk, but it would only work doing it a very long way.

awk ' $2<=0.05' file_1.txt > file_2.txt
awk ' $4<=0.05' file_2.txt > file_3.txt

etc, and achieved the desired result

96.2    0.003   89.4    0.001   106.9   0.000
105.8   0.003   72.6    0.003   86.7    0.002
192.4   0.001   312.8   0.001   274.3   0.001

but my file is 198 columns and 57000 lines

I also tried piping the awk commands together with no luck. it only searches $2

awk ' $2<=0.05 || $4=<0.05' File_1.txt > File_2.txt

74.9    0.006   59.6    0.051   72.2    0.002
96.2    0.003   89.4    0.001   106.9   0.000
105.8   0.003   72.6    0.003   86.7    0.002
192.4   0.001   312.8   0.001   274.3   0.001

I’m pretty new at this and would appreciate any advice on how to achieve this using awk

Thanks

Sam

Answer 1

Perhaps this is what you’re looking for. It will search every even numbered column and check that each of these columns contain numbers smaller than ‘0.05’:

awk 'NF>1 { for(i=2;i<=NF;i+=2) if ($i>0.05) next }1' File_1.txt

Results:

96.2    0.003   89.4    0.001   106.9   0.000
105.8   0.003   72.6    0.003   86.7    0.002
192.4   0.001   312.8   0.001   274.3   0.001