Question

I’m trying to upload files to Amazon S3 using AWS::S3, but I’d like to compress them with Zlib first. AWS::S3 expects its data to be a stream object, i.e. you would usually upload a file with something like

AWS::S3::S3Object.store('remote-filename.txt', open('local-file.txt'), 'bucket')

(Sorry if my terminology is off; I don’t actually know much about Ruby.) I know that I can zlib-compress a file with something like

data = Zlib::Deflate.deflate(File.read('local-file.txt'))

but passing data as the second argument to S3Object.store doesn’t seem to do what I think it does. (The upload goes fine but when I try to access the file from a web browser it doesn’t come back correctly.) How do I get Zlib to deflate to a stream, or whatever kind of object S3Object.store wants?

Answer 1

I think my problem before was not that I was passing the wrong kind of thing to S3Object.store, but that I was generating a zlib-compressed data stream without the header you’d usually find in a .gz file. In any event, the following worked:

str = StringIO.new()
gz = Zlib::GzipWriter.new(str)
gz.write File.read('local-file.txt')
gz.close

AWS::S3::S3Object.store('remote-filename.txt', str.string, 'bucket')