When I upload a file into django, it first goes through the UploadHandlers. i have used this to write a progress bar. when the upload is complete, the ‘upload file’ view gets called. The request object will have either an InMemoryUploadedFile or a temporary file on disk.
Here is my question: The docs suggest to copy the file chunk by chunk. For a large file, would this not take a long time (with the progress bar stuck at 100%)?
Would it not be better to do a ‘move’ operation? eg on windows you can move a file almost instantly within the same drive letter, copying takes a long time. i believe the same applies on unix systems.
Yes, the same applies to Unix systems… where a “drive letter” is more of a path, so you might find that /opt/django/YourApp and /opt/django/tmpUploadDir are actually two different devices.
So, trusting on “move” it’s not really worth it, by and large. On the other hand, you may do your copying outside the client viewing process, after he/she has dismissed the “100% complete” dialog.
Or you might factor the copy time in the total percentage. Say that you know that copy speed is 1 Mb per second, and you observe network upload speed proceeding at 100 Kb per second. Then you know that for a 1 GB file total transfer time will be 1100 seconds (not 1000), and once calculated that the “old” percentage is X, you will be able to display it as X*DiskSpeed/(DiskSpeed + NetworkSpeed). When the user has uploaded 100% of the file, he will see the progress bar at 91%, growing until it reaches 100% when copy is also complete.
This last method can be used at advantage if your processing is more than a simple copy, e.g. if you are recoding a video through ffmpeg. You do need a good guesstimate of the processing time beforehand, though, otherwise you’ll end up with an “upload” bar that, while nonmonotonic (i.e. you should never see it going from 80% to 79%), will progress in fits and starts.
My opinion is that compared to upload time, copy time is negligible and may be safely ignored. If it is not, then maybe displaying a second, different progress bar might be in order, so that the user doesn’t see anything he/she might believe “anomalous”.