We have an extensive existing codebase and we’ve added load-balanced servers with a single master server to the equation now. There are various apps that contain models with uploaded files and images which all work fine… However, this raises the obvious problem of the rsync delay. Rsync is in the crontab and set to run every minute but this still means there’s a potential 59 second wait between content being created and it actually existing on the webservers.
What I would like, is to be able to register some kind of ‘post file changed’ handler that triggers rsync whenever a new file is uploaded. I can’t find anything of the sort though! Django has file upload handlers, but these appear to only deal with the actual upload stream, not the file as it is saved to the filesystem thereafter.
The best approach I can see is to create simple extensions to FileField, FieldFile, ImageField and ImageFieldFile as part of my project and hook into the save and delete methods in the FileField. Essentially, to create custom File and Image fields with this behaviour added. This isn’t massively complicated to do but it doesn’t seem like the most elegant solution to me. I’ll need to teach South about my new fields, update every model that is affected and then create hordes of south migrations (which I’m pretty sure will clash with some code we have pending).
I’m also looking into creating a custom Storage class for the project, but I’m nervous about this having far-reaching effects on other pieces of code.
I can’t believe no-one has come across this issue before, is there a canonical approach?
Thanks very much!
If you want to tackle this problem from the server-side (eg. similar solution to rsync) and you’re running Linux, you might want to check out lsyncd:
http://code.google.com/p/lsyncd/
lsyncd uses inotify in the Linux kernel to watch directories and invoke an rsync as soon as files are modified. Fairly simple to drop in.