Question

I have files with multiple columns and I want to read values from a particular column. I can read a column using awk{print $column_number}.

Each file has different lengths of columns i.e. some might be 1000 entries long and others might be just 2 and so on. The entries themselves range from 1 digit up to a maximum of 5 digits. This is the same for all the files.

I want to count the range of the most repeated values. For example if the column reads:

5
93
201
2002
20003
20005
20087
31450
31451
31452
31458
52400
52428

then I want to store 31,400 as the most repeated value, then 20,000 and 52,000 as the second most and third most repeated values and so on. You can say that I am rounding up the values to see the most repeated numbers, if that makes sense. These values(the most repeated, the second most repeated) can be considered to be multiples of 100. So basically the code should look something like this:

for f in ls path-to-the-files/*

do

while read i

    do
    <do the operation to sort and store the values>
done        

done

I’d appreciate help with this!

Answer 1

This might work for you:

sed 's/.\?.$//;s/^$/0/;s/.$/,&00/;s/^,/0,/' file | sort | uniq -c | sort -nr 
4 31,400
3 20,000
2 52,400
2 0,000
1 2,000
1 0,200

If you’re not interested in the , format use:

sed 's/.\?.$//;s/$/00/;s/^00$/0/' file | sort | uniq -c | sort -nr
  4 31400
  3 20000
  2 52400
  2 0
  1 2000
  1 200