Home/ Questions/Q 7830603
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-02T11:18:43+00:00

Based on this question Group files and pipe to awk command I have a

  • 0

Based on this question Group files and pipe to awk command

I have a set of files like this:-

-rw-r--r-- 1 root root 497186 Apr 21 13:17 2012_03_25
-rw-r--r-- 1 root root 490558 Apr 21 13:17 2012_03_26
-rw-r--r-- 1 root root 488797 Apr 21 13:17 2012_03_27
-rw-r--r-- 1 root root 316290 Apr 21 13:17 2012_03_28
-rw-r--r-- 1 root root 490081 Apr 21 13:17 2012_03_29
-rw-r--r-- 1 root root 486621 Apr 21 13:17 2012_03_30
-rw-r--r-- 1 root root 490904 Apr 21 13:17 2012_03_31
-rw-r--r-- 1 root root 491788 Apr 21 13:17 2012_04_01
-rw-r--r-- 1 root root 488630 Apr 21 13:17 2012_04_02

Based on the answer in the linked question I have a script with the following code, which works fine:-

DIR="/tmp/tmp"
for month in $(find "$DIR" -maxdepth 1 -type f | sed 's/.*\/\([0-9]\{4\}_[0-9]\{2\}\).*/\1/' | sort -u); do
  echo "Start awk command for files $month"
  power=$(awk -F, '{ x += $1 } END { print x/NR }' "$DIR/${month}"_[0-3][0-9])
  echo $power
done

The below command on it’s own returns a list like this:-

find /tmp/tmp -maxdepth 1 -type f | sed 's/.*\/\([0-9]\{4\}_[0-9]\{2\}\).*/\1/' | sort -u

2011_05
2011_06
2011_07
2011_08
2011_09
2011_10
2011_11
2011_12
2012_01
2012_02
2012_03
2012_04

The find command is passing a set of files using a GLOB to AWK to be processed as a batch.

Based on this, i want to be able to run the following cut commands

head -1 FirstFile | date -d "`cut -d, -f7`" +%s

tail -1 LastFile | date -d "`cut -d, -f7`" +%s

These need to be run for the FIRST and LAST file PER SET

So for 2012_03 above, the head would need to be run for the 2012_03_25 file and the tail would need to be run for the 2012_03_31 as these are the first and last files in the set for March.

So basically I need to be able to get the FIRST and LAST file PER BATCH.

I hope I have made this clear enough, if not please comment.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team
    DIR="/tmp/tmp"
for month in $(find "$DIR" -maxdepth 1 -type f | sed 's/.*\/\([0-9]\{4\}_[0-9]\{2\}\).*/\1/' | sort -u); do
      echo "Start awk command for files $month"
      IFS=, read start end power < <(awk -F, 'BEGIN{OFS = ","} NR == 1 {printf "%s,", $7} { x += $1; d = $7 } END { print d, x/NR }' "$DIR/${month}"_[0-3][0-9])
      echo $power
      date -d "$start" +%s
      date -d "$end" +%s
done

    Here is how you would use a here-doc, which should work in most shells:

          read start end power <<EOF
$(awk -F, 'NR == 1 {printf "%s ", $7} { x += $1; d = $7 } END { print d, x/NR }' "$DIR/${month}"_[0-3][0-9]))
EOF
    • 0