Question

I have a large txt file space delimited which I split into 18 smaller files (each with their own number of columns). This split is based on a delimiter i.e. whenever the timestamp hits midnight. So effectively, I’ll end up with a 18 files in the form of (note, ignore the dashes and pipes, I’ve used them to improve the readability):

file1

time ----------- valueA - valueB  
12:00:00 AM |      54.13 | 239.12  
12:00:01 AM |      51.83 | 119.93  
..  

file18

time ---------- valueA - valueB - valueC - valueD  
12:00:00 AM |      54.92 | 239.12  | 231.23  | 882.12  
12:00:01 AM |      23.92 | 121.92  | 201.23  | 892.12   
..  

Once I split the file I then perform some processing on each of the files using AWK so in short there’s 2 stages the ‘split stage’ and the ‘processing stage’.

Unfortunately, the timestamp contained in the large txt file is in 1 of 2 formats. Either the desirable 24 hour format of “00:00:01” or the undesirable 12 hour format of “12:00:01 AM”.

As a result, I’m trying to convert all formats to be 24 hours and I’m not sure how to do this. I’m also not sure whether to attempt this at the split stage using bash or at the process stage using AWK. I know that the following function converts 12 hour to 24 hr

'date --date="12:00:01 AM" +%T'

however, I’m not sure how to incorporate this into my shell script were I’m using ‘while read line’ at the ‘split stage’ or whether I should do the time conversion in AWK (if possible?) at the ‘processing stage’.

Answer 1

see the test below, is it helpful for you?

 kent$  echo "12:00:00 AM |      54.92 | 239.12  | 231.23  | 882.12  "\
|awk -F'|' 'BEGIN{OFS="|"}{("date --date=\""$1"\" +%T") |getline $1;print }'

output

 00:00:00|      54.92 | 239.12  | 231.23  | 882.12