For the sake of brevity consider a facebook style image content serving app. Users can upload content as well as access content shared by other people. I am looking at best ways of handling this kind of file serving application through Java servlets. There is surprisingly little information available on the topic. I’d appreciate if someone can tell me their personal experiences on a small setup (a few hundred users).
So far I am tempted to use the database as a file system (using mongodb) but the approach seems cumbersome and tedious and will need replicating part of the functionality already provided by OS native filesystems. I don’t want to use commercial software or have the bandwidth to write my own like facebook. All I want is to be able to do this through free software on a small server with a RAID or something similar. A solution that scales well to multiple servers would be a plus. The important thing is to serve it through java servlets (I am willing to look into alternatives but they have to be usable through java).
I’d appreciate any help. Any references to first hand experiences would be helpful as well. Thanks.
Guru –
I set up something exactly like this for members of my extended family to share photos. It is a slightly complicated process that includes the following:
1) Sign up for Amazon Web Services, notably their S3 (Simple Storage Service). There is a free storage tier that should cover the amount of users you described.
2) Set up a web application that accepts uploads. I use Uploadify in combination with jQuery and ajax, to upload to a servlet that accepts, scans, logs, and does whatever else I want with the file(s). On the servlet side, I use ESAPI’s upload validation mechanism, part of the validation engine, which is just built on top of Commons File Upload, which I have also used by itself.
3) After processing the file(s) appropriately, I use JetS3t as my Java-AmazonS3 API and upload the file to Amazon S3. At that point, users can download or view photos depending on their level of access. The easiest way I have found to do this is to use JetS3t in combination with the Web Application Authentication to create Temporary URL’s, which give the user access to the file for a specific amount of time, after which the URL becomes unusable.
A couple of things, if you are not concerned with file processing and trust the people uploading their files completely, you can upload directly to Amazon S3. However, I find it much easier to just upload to my server and do all of my processing, checking, and logging before taking the final step and putting the file on Amazon S3.
If you have any questions on the specifics of any of this, just let me know.