Question

Possible Duplicate:
How to download a file in python

I’m playing with Python for doing some crawling stuff. I do know there is urllib.urlopen("http://XXXX") That can help me to get the html for target website. However, The link to the original image in that webpage will usually make the image in the backup page unavailable. I am wondering is there a way that can also save the image in the local space, then we can read the full content on the website without internet connection. It’s like back up the whole webpage, but I’m not sure is there any way to do that in Python. Also, if it can get rid of the advertisement stuff, it will be more awesome though. Thanks.

Answer 1

If you’re looking to backup a single webpage, you’re well on your way.

Since you mention crawling, if you want to backup an entire website, you’ll need to do some real crawling and you’ll need scrapy for that.

There are several ways of downloading files off the interwebs, just see these questions:

1. Python File Download
2. How to- download a file in python
3. Automate file download from http using python

Hope this helps