Home/ Questions/Q 6121673
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-23T15:48:39+00:00

Going to use Apache Nutch v1.3 to extract only some specific content from the

  • 0

Going to use Apache Nutch v1.3 to extract only some specific content from the webpages. Checked parse-html plugin. Seems it normalizes each html page using tagsoup or nekohtml. This is good. I need to extract only text inside <span class='xxx'> and <span class='yyy'> elemetns on the web-page. Would be great if extracted texts are saved into different fields (e.g. content_xxx, content_yyy).
My question is: should I write my own plugin or this could be done using some standard way?

The best way would be apply XSLT on normalized web-page and get the result. Is that possible?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Building your own ParsingFilter and IndexingFilter is easy. Nutch provides you with the DOM document, which you only need to traverse and search for your div. Then you simply add the new fields to your index and schema and your done.

    There are some examples on how to do this:

    http://wiki.apache.org/nutch/HowToMakeCustomSearch

    http://sujitpal.blogspot.com/2009/07/nutch-custom-plugin-to-parse-and-add.html

    Good luck

    • 0