Home/ Questions/Q 7933357
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-03T21:18:29+00:00

I’ve been using Scrapy for about 1 week now, and want to store the

  • 0

I’ve been using Scrapy for about 1 week now, and want to store the images to amazon S3, and they mentioned that they support images uploading to amazon S3 but it’s not documented. So does anyone know how to use Amazon S3 with Scrapy?

Here’s their Scrapy documentation for media pipeline.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You need 3 settings:

    AWS_ACCESS_KEY_ID = "xxxxxx"
AWS_SECRET_ACCESS_KEY = "xxxxxx"
IMAGES_STORE = "s3://bucketname/base-key-dir-if-any/"

    that’s all, ie. images will be stored using same directory structured described at http://readthedocs.org/docs/scrapy/en/latest/topics/images.html#file-system-storage, ie:

    s3://bucketname/base-key-dir-if-any/full/3afec3b4765f8f0a07b78f98c07b83f013567a0a.jpg
    • 0