Home/ Questions/Q 4595474
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-21T22:58:32+00:00

I have a cron job scrape.sh that look like this: #!/bin/bash touch rage cd

  • 0

I have a cron job scrape.sh that look like this:

#!/bin/bash
touch rage
cd /etc/myproject/scraper
scrapy crawl foosite --set FEED_URI=../feeds/foosite.xml --set FEED_FORMAT=xml
scrapy crawl barsite --set FEED_URI=../feeds/barsite.xml --set FEED_FORMAT=xml

When it executes the file rage does get created and judging from my syslog it does run as root, so permissions shouldn’t be a problem.

May  6 17:35:01 server CRON[10233]: (root) CMD (/etc/myproject/scraper/scrape.sh)
May  6 17:40:01 server CRON[17804]: (root) CMD (/etc/myproject/scraper/scrape.sh)

When I run scrape.sh it executes as expected and puts the foosite.xml file in the ../feeds directory, the directory exist and is empty when the cron jobs starts. What can I do to solve this issue?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team
    1. If I were going to guess the problem
      it was an environment issue (e.g.
      scrapy is not in the path).
    2. To debug, make sure your cron job is sending the standard out and standard error to a log file/and or syslog
    • 0