Question

I’m trying to write a basic synchronisation program to copy files between two directories.

The plan is to have some sort of database of the files which were already copied so when the program scans through the directory it can pick out what it has already copied and what it needs to copy again, i.e. modified or new files.

One of the problems which I’m not sure how to approach is how should I deal with files which are renamed by the user. How can i tell that a renamed file is actually the same file as one listed in the database, albeit with a different name. Is there an underlying file ID that the program could extract? I’d like the code to be portable so that probably makes this more difficult considering filesystems will be different.

I was thinking that I could make note of the size and creation date of each file and keep that information in the database to determine if the file was actually just renamed. If two files have the same information size/date I could store a hash or something to tell them apart but I dont know how efficient that’d be.

Any suggestions? (I’m using C++/QT)

Answer 1

To track changes to files in a given folder while your application is running, see QFileSystemWatcher, with which you can track all file changes within a directory after calling QFileSystemWatcher::addPath("directory/to/watch/"):

The fileChanged() signal is emitted when a file has been modified,
renamed or removed from disk. Similarly, the directoryChanged() signal
is emitted when a directory or its contents is modified or removed.
Note that QFileSystemWatcher stops monitoring files once they have
been renamed or removed from disk, and directories once they have been
removed from disk.

In order to keep tracking the files once they have been renamed, just re-enable watching them using QFileSystemWatcher::addPath() again.

---

When your application wasn’t running during the rename operation, you have to watch the contents of the file (if you also want to cover the case when a file has been deleted and copied back, where the inode on linux / whatever on other file system types has changed).

This can be done using MD5 sums, but it requires your application to read the whole file, which can be slow for files greater than, let’s say, 10 megabytes. If this drawback is acceptable, just save the MD5 sum returned by QCryptographicHash::hash(file.readAll(), QCryptographicHash::Md5). Of course, there are some corner cases where you have hash collisions, but for most applications this should not be a problem. But please note, that it can happen that you will not detect a change of the contents. Also note that the simple one-liner above is blocking and reads the whole file into memory before computing the MD5 sum. Use the step-wise MD5 sum calculation provided by QCryptographicHash (using addData() and result() instead) to “stream” the file into the MD5 calculation.

Comparing the file sizes before reading the whole contents instead of creating the MD5 sum will work in most cases (it happens not too often that a change results in the same file size; and in such cases you could fall back to the MD5 sum comparison). But after you noticed a change, you need to read the MD5 sum anyway, to be able to detect future changes!