Question

I am using Python Boto to download all the files in a bucket to my windows system. The problem is that there are subdirectories on the bucket (or whatever you call them). This is my partial code which fails with obvious error that its trying to write to a path that does not exist since it maps S3 path to windows.

logsBucket = s3.get_bucket('mylogs')

for b in logsBucket:
    #Download log
    print b.name
    key = logsBucket.get_key(b.name)
    fp = open("D:\\mylogs\\" + b.name, "w")
    key.get_file(fp)

The key is something like this ‘mysite/access/*.gz’

I don’t know I could be sleepy or I’ll just do this on my linux system, but doing this on windows will be more useful to me.

Answer 1

Yep sleepy! I found the answer in random google search of an example script wpstorm.net/2010/11/aws-s3-logs-boto-python This shows how to actually deal with s3 bucket correctly. I should have been using thr bucket.list with prefix. Makes sense now. Hope it helps someone else