Is there a module out there that can give me links to all the pages a website has??
Why I need it: I want to crawl some sites and search for tags in them, searching only on mainpage is not enough.
Thanks,
Share
Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
Is there a module out there that can give me links to all the pages a website has??
Why I need it: I want to crawl some sites and search for tags in them, searching only on mainpage is not enough.
Thanks,
HTML::SimpleLinkExtor is a bit simple than HTML::LinkExtor. You might check out my half-hearted attempt at my webreaper tool that has some of the code that you’ll probably need.