Question

I have a server with an open port which receives between 50 and 1000 messages per second. By message I mean that a single line of text is sent.

Essentially we want to record these messages in a file which will be processed every hour (or x minutes).

I have created a bash script (see below) which runs in the background and it works except when I kill the socat process (so I can take the file for processing and it can start a new file) we get part of a message, plus I am sure we are losing messages during the split second that socat is down.

DELAY="3600"
while true
do  
    NEXT_STOP=`date +%s --date "$DELAY second"`
    (
        while [ "$(date +%s)" -lt "$NEXT_STOP" ]
        do
            killall socat
            socat -u TCP-LISTEN:6116,reuseaddr,keepalive,rcvbuf=131071,reuseaddr OPEN:/var/ais/out_v4.txt,creat,append
        done
    ) & sleep $DELAY ; killall socat

    mv /var/ais/out_v4.txt "/var/ais/_socat_received/"$(date +"%Y-%m-%d-%T")"__out_v4.txt"
done

Is there a way to:

1. Get socat to rotate its output file without killing the process
2. Or can we purge the content of the file whilst SOCAT is writing to it. e.g. cut the first 10000 lines into another file, so the output file remains a manageable size?

Many thanks in advance

Answer 1

For anyone interested the final solution looks like the following, the key difference to Nicholas solution below is that I needed to grep the PID of the socat process rather than use $?:

#!/bin/bash
DELAY=600
SOCAT_PID=$(/bin/ps -eo pid,args | grep "socat -u TCP-LISTEN:12456" | grep -v grep | awk '{ print $1 }')

while `kill -0 $SOCAT_PID`
do  
  touch /var/ais/out.txt
  NEXT_STOP=`date +%s --date "$DELAY second"`
  while  `kill -0 $SOCAT_PID` && [ "$(date +%s)" -lt "$NEXT_STOP" ]
  do
    head -q - >> /var/ais/out.txt
  done
  mv /var/ais/out.txt "/var/ais/_socat_received/"$(date +"%Y-%m-%d-%T")"__out.txt"
done

In addition adding the start script within an infinite while loop so that when the client disconnects we restart socat and wait for the next connection attempt:

while true
do
socat -u TCP-LISTEN:12456,keepalive,reuseaddr,rcvbuf=131071 STDOUT | /var/ais/socat_write.sh
done