Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
In Groovy, how do I grab a web page and remove HTML tags, etc., leaving only the document’s text? I’d like the results dumped into a collection so I can build a word frequency counter.
Finally, let me mention again that I’d like to do this in Groovy.
Assuming you want to do this with Groovy (guessing based on the groovy tag), your approaches are likely to be either heavily shell-script oriented or using Java libraries. In the case of shell-scripting I would agree with moogs, using Lynx or Elinks is probably the easiest way to go about it. Otherwise have a look at HTMLParser and see Processing Every Word in a File (scroll down to find the relevant code snippet)
You’re probably stuck with finding Java libs for use with Groovy for the HTML parsing, as it doesn’t appear there are any Groovy libs for it. If you’re not using Groovy, then please post the desired language, since there are a multitude of HTML to text tools out there, depending on what language you’re working in.