Question

I’m trying to access the http://s3.amazonaws.com/commoncrawl/parse-output/segment/ bucket with boto. I can’t figure out how to translate this into a name for boto.s3.bucket.Bucket().

This is the gist of what I’m going for:

s3 = boto.connect_s3()
cc = boto.s3.bucket.Bucket(connection=s3, name='commoncrawl/parse-output/segment')
requester = {'x-amz-request-payer':'requester'}
contents = cc.list(headers=requester)
for i,item in enumerate(contents):
    print item.__repr__()

I get “boto.exception.S3ResponseError: S3ResponseError: 400 Bad Request … The specified bucket is not valid…”

Answer 1

The bucket name would be commoncrawl. Everything that appears after that is really just part of the name of the keys that appear in the bucket.