Question

Hi i am working on scrapy, i created a scrapy folder with scrapy startproject example
and written spider to scrape all the data from the url, and
I had run the spider using the command scrapy crawl spider_name, its working fine and able to fetch data.

But i had a requirement that i need to run the scrapy with a single spider file created i mean a single py file something like

python -u /path/to/spider_file_inside_scrapy_folder_created.py

Is it possible to run a spider without scrapy crawl command after creating a scrapy project folder with spider.py file

Answer 1

Try the runspider command:

scrapy runspider /path/to/spider_file_inside_scrapy_folder_created.py