Question

I have a script that runs every minute at XX:00. The script loops over all subfolders in a given directory and performs operations on the files inside;

folder=/path/to/directory #Starting directory
someerror=0 #Did we have an error?

#CD to directory. Does it even exist?
cd $folder
RETVAL=$?
[ $RETVAL -eq 0 ] && echo Success changing directory to $folder && mainfolderexist=1
[ $RETVAL -ne 0 ] && echo Failure changing directory to $folder && mainfolderexist=0

if [ $mainfolderexist -eq 1 ]; then
    shopt -s nullglob
    for dir in $folder/*/
    do
    thedirname=`basename $dir` #Get directory name
    #cd to sub dir. Does it even exist?
    cd $dir
    RETVAL=$?
    [ $RETVAL -eq 0 ] && echo Success changing directory to $dir && subfolderexist=1
    [ $RETVAL -ne 0 ] && echo Failure changing directory to $dir && subfolderexist=0
    if [ $subfolderexist -eq 1 ]; then
        #perform some operation on all files in this directory
        someApp -someArgs --name=$thedirname *
    else #sub folder doesn't exist
        someerror=1
        break
    fi

    #next folder
    done
else #main folder doesn't exist
    someerror=1
fi

#REPEAT (only if no errors occured)
if [ $someerror -eq 0 ]; then
at now + 1 minutes << END
/bin/bash "$0" "$@"
END
fi

The way I use this, is I upload directories to the server using SFTP, to a folder like /home/incoming, and once the directory is fully uploaded I will move it to /path/to/directory. Now this is the part I am worried about.

So far I’ve been making sure to only move directories between XX:XX:02 and XX:XX:50, but is this even neccesary? I would like to automate the upload+move process without taking system time into account so I am wondering;

1. What if a directory is in the process of getting moved (mv "somedir" "/path/to/directory/somedir") at XX:00 and the script runs, looping over all directories?
2. What if the system loses power during the mv command? If the directory will end up half moved or something similar, I will have to write a script verifying this before executing the above script.

Answer 1

If your source and destination paths are on the same filesystem, then mv is an atomic operation. Since it does not actually involve copying or otherwise relocating files, your directories will never end up in a “half-moved” state.

If, on the other hand, your source and destination paths are on different filesystems, then mv is actually a copy followed by a delete over the entire tree, which can take a substantial amount of time and, if interrupted, will leave things in a half-completed state.