Question

In Ubuntu if a file /A/largefile.foo is moved via ‘cp’ or ‘mv’ to /B/largefile.foo, how can I ensure that the entire file is written before attempting to move the file a second time to /C/largefile.foo?

Case:
1. An remote automated Script X begins an rsync to destination folder A of a very large file.
2. A cron job on the local machine hourly moves files from folder A to folder B, but I do not want to copy partial files (I only want files that have finished uploading completely)

Case:
1. A cron job periodically moves files from folder B to folder C, but I do not want to copy partial files (I only want files that have finished uploading completely)

Note:
I’m looking for a solution that does not require the addition of pid files

Answer 1

For the first rsync case, you can use the --partial-dir option combined with --delay-updates, it will write partial files to a separate location, renaming them into place once done.

You should make sure that the partial directory is on the same filesystem as the destination, so the renames will be atomic.

for the second case use mv (again assuming one file system) which will be atomic.

see the rsync manual for more information