Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
Alright, I’ve been messing around with Nutch and need to know what parameter inside the crawl-urlfilter.txt file I edit so the spider has no boundaries. In other words I want it to roam around the web outside of a specified domain.
I’m assuming it has to do with this line, but I don’t know how to edit it correctly to do as I want it to:
+^http://([a-z0-9]*\.)*urlz.net/
I’m not framiliar with Nutch but this is just a regular expression.
Would probably work just fine, or some variation thereof. Its just matching a pattern. The one I just wrote above should match anything starting with http:// and then any number of letters, numbers or dots.