Question

We have a web application which needs to store uploaded images with EdgeCast or Amazon S3. To do so we need to re-upload those images to EdgeCast / S3 first. There are a number of options I can think of:

1. Do an inline curl re-upload, which will upload the image using the desired EdgeCast / S3 API. The disadvantage is that with concurrent uploads the server load will be massive.
2. Queue re-uploads. I know it is possible but I have no idea how to do it.
3. Don’t bother with the re-uploads and upload directly to EdgeCast / S3 from the client end.
4. Mount EdgeCast’s / S3’s FTP into the filesystem, and then just copy uploaded files, letting the FTP daemon do the rest.

Which is the best solution, and are there any other ones? I suspect it is either 1 or 2.

Edit: My application is in PHP

Answer 1

Not sure about EdgeCast but I know with Amazon S3 the best way to do this was to POST the file directly to the file server. See http://doc.s3.amazonaws.com/proposals/post.html

Doing it this way you would provide a HTML FORM with some field like folder id, file name, public key, timestamp etc to ensure it was secure and only you could upload to the server. The server would redirect their browser once the upload was complete from that page they redirected to you could inspect the query string to find out if the upload was successful or not then record the FileID into your DB.

Works great for reducing load on your server and gives the user a faster experience, however it can lead to orphan files if the upload succeeds but the DB insert fails.