We make distributal software that stores some data (attachments) in a) a database or b) Amazon S3. The database is used because it requires no other configuration. Amazon S3 is the better option.
What we want now is a solution for customers that don’t want to use Amazon S3. We can obviously just use the filesystem but this can be problematic if there are multiple web servers and the files need to be replicated; it also requires us to write extra code to handle the various permuations of problems that can happen.
What we would prefere is if there was a piece of server software that essentially replicates Amazon S3’s API. That way our clients can install the server on a box; and we don’t need to change any code. So … is there any such software out there?
This is possible via OpenStack Object Storage (code-named Swift), which is open source software for creating redundant, scalable object storage using clusters of standardized servers, specifically its recently added (optional) S3 API layer, which emulates the S3 REST API on top of Object Storage.
See Configuring Object Storage with the S3 API for the official documentation – a more insightful and illustrated small tutorial regarding the entire setup is available in S3 APIs on OpenStack Swift (which builds on the more complex Installing an OpenStack Swift cluster on EC2 though).
Update
An noteworthy alternative is Ceph, which is a unified, distributed storage system designed for excellent performance, reliability and scalability – interestingly it provides all three common storage models, i.e. Object Storage, Block Storage and a File System and the RADOS Gateway provides Amazon S3 and OpenStack Swift compatible interfaces to the RADOS object store [emphasis mine], see RADOS S3 API for details on currently supported S3 API features.