Question

I have a script that dumps the db and copies it to S3. This is the crontab entry that calls the script:

*/1 * * * * /root/scripts/backupDB.sh

backupDB.sh:

#!/bin/sh -e

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/

date_now=`date +%Y_%m_%d_%H_%M`
dir_name="db_backup"
tar_name="db_backup_${date_now}.tar.gz"
file_name="${dir_name}/${tar_name}"

if [ -z "$dir_name" ]; then
    mkdir $dir_name
fi

log() {
    echo $1
    echo $1 >> ${HOME}/tmp/mybackup.log
}

do_cleanup(){
    rm -rf db_backup* 
    log 'cleaning up....'
}

do_backup(){
    log 'Starting the backup' && \
    /usr/bin/mongodump -o ${HOME}/${dir_name} && \
    log 'Data dump created'
    /bin/tar -cjf ${HOME}/${tar_name} ${HOME}/${dir_name} && \
    log 'Created archive'

    log 'saving the backup archive in amazon S3' && \
    /usr/bin/s3cmd put --acl-private ${HOME}/${tar_name} s3://bucket/db-backups/${tar_name} && \
    log 'data backup saved in amazon s3'
}


do_backup && do_cleanup

If run through the terminal, the same script dumps all the databases. If run through cron, only dumps a few collections of each database. The mongodb log is pasted here. It shows nothing weird. You can see connections being made at the beginning of every minute, correspond to the crontab entry that calls the script. But it also shows that the connection is ended immediately. What might be the problem?

Answer 1

How long does the script take to run?

Since you’re running it from cron rather frequently, there’s always a worry that the next instance might start before the first has finished leading to all sorts of issues with the output depending on what the script does.

You’ll want to implement some form of locking to prevent concurrent runs.