Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
It’s very easy to fetch the simple web page.
As I can see from python’s manumal
import urllib2
response = urllib2.urlopen('http://python.org/')
html = response.read()
But how to fetch all site?
Can anybody please provide me the code?
Use BeautifulSoup for parsing the site and repeat the process for every link unless it leads you outside of the domain.
Quite straightforward, but it gets complex if you try to fetch also the dynamic content, that does not have links leading to it.