Home/ Questions/Q 7735293
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-01T07:27:54+00:00

I’ve noticed a difference between the returns from boto’s api depending on the bucket

  • 0

I’ve noticed a difference between the returns from boto’s api depending on the bucket location. I have the following code:

con = S3Connection(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
bucket = con.get_bucket(S3_BUCKET_NAME)
keys = bucket.list(path)
for key in keys:
  print key

which im running against two buckets, one in us-west and one in ireland. Path in this bucket is a sub-directory, against Ireland I get the sub directory and any keys underneath, against us-west I only get the keys beneath.

So Ireland gives:

<Key: <bucketName>,someDir/>
<Key: <bucketName>,someDir/someFile.jpg>
<Key: <bucketName>,someDir/someOtherFile.jpg>

where as US Standard gives:

<Key: <bucketName>,someDir/someFile.jpg>
<Key: <bucketName>,someDir/someOtherFile.jpg>

Obviously, I want to be able to write the same code regardless of bucket location. Anyone know of anything I can do to work around this so I get the same predictable results. Or even if it’s boto causing the problem or S3. I noticed there is a different policy for naming buckets in Ireland, do different locals have their own version of the api’s?

Thanks,

Steve

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Thanks to Steffen, who suggested looking at how the keys are created. With further investigation I think I’ve got a handle on whats happening here. My original suposition that it was linked to the bucket region was a red herring. It appears to be due to what the management console does when you manipulate keys.

    If you create a directory in the management console it creates a 0 byte key. This will be returned when you perform a list.

    If you use boto to create/upload a file then it doesn’t create the folder. Interestingly, if you delete the file from within the folder (from the AWS console) then a key is created for the folder that used to contain the key. If you then upload the bey again using boto, then you have exactly the same looking structure from the UI, but infact you have a spurious additional key for the directory. This is what was happening to me, as I was testing our application I was clearing out keys and then finding different results.

    Worth knowing this happens. There is no indicator in the UI to show if a folder is a created one (one that will be returned as a key) or an interpreted one (based on a keys name).

    • 0