Question

I am working on a Rails web application, running on a Heroku stack, that handles looking after some documents that are attached to a Rails database object. i.e. suppose we have an object called product_i of class/table Product/products, and product_i_prospectus.pdf is the associated product prospectus, where each product has a single prospectus.

Since I am working on Heroku, and thus do not have root access, I plan to use Amazon S3 to store the static resource associated with product_i. So far, so good.

Now suppose that product_i_attributes.txt is also a file I want to upload, and indeed I want to actually fill out information in the product_i object (i.e. the row in the table corresponding to product_i), based on information in the file product_i_attributes.txt.

In a sentence: I want to create, or alter, database objects, based on the content of static text files uploaded to my S3 bucket.

I don’t actually have to be able to access them once they are in the bucket strictly speaking, I just need to create some stuff out of a text file.

Answer 1

I have done something similar with csv files. I would not try to process the file directly at upload as it can be resource intensive.

My solution was to upload the file to s3 and then call a background job method(delayed_job, resque, etc.) that processed the csv after upload. You could then call a delete after the job processed to remove the file from s3 if you no longer needed it after processing.

For Heroku this will require that you add a worker (if you don’t already have one) to process the background jobs that will process the text files.