Question

In the AWS-SDK, it lists ruby code of the form:

 s3 = AWS::S3.new
 bucket = s3.buckets[bucket_name]
 tree = bucket.as_tree(:prefix => 'myshop/products')
 directories = tree.children.select(&:branch?).collect(&:prefix)

fail error: “Unable to find marker in S3 list objects response”

Structure of directory

/myshop/products/1474472/original.jpg
/myshop/products/1474472/small.jpg
/myshop/products/1474472/mini.jpg
/myshop/products/1333333/original.jpg
/myshop/products/1333333/small.jpg
/myshop/products/1333333/mini.jpg

…

more of 100 000 obj

I want to verify that the directory(for example “1474472”) was created

my plan: aws-s3-list-> ruby-array->find in array (array.include?)

!!!need very fast method – soon the end of the world 🙂

Answer 1

There is no such stuff as folders in Amazon S3. It is a “flat” file system. Have a look into this answer.

What you really are looking for is verifying whether a given prefix (“/myshop/products/1474472”, for instance) exists in your bucket.
Their REST API definitely supports it, have a look into the documentation. You need to list the keys (which would be the “file names”) matching a given prefix, that can be passed as parameter. You can also optimize your call by setting the max-keys parameter to 1. That way, if you receive any non-zero amount of items in the response, the bucket already contains files with names starting with the given prefix.