Home/ Questions/Q 4004724
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-20T08:18:28+00:00

The best example I saw so far is the http://www.instapaper.com/ . They can get

  • 0

The best example I saw so far is the http://www.instapaper.com/ . They can get the text from any page.

In my case, I need to get the text and also generate a list considering that I will have one page with the news list of each site.

For example, nytimes.com (just an example). I have to get all links and get the text if it exists. Also, maybe I need to specify some URL criteria, like generate feeds from links where the url contains something like “/[year]/[month/[day]/[category]/post-name”.

I don’t want the complete code, just the concept and best approach. Any ideias?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    This is painful but the only good solution is to use an HTML parser and parse all the hrefs. I recommend using a library that allows you to easily select all hrefs. I have heard of this one http://code.google.com/p/phpquery/ but never used it. What you would do is load each page and then select all hrefs.

    There is really no easier way. If you changed your technology to something like java or python, then you can leverage multi-thread power and speed up the process. Of course once you analyze, save the data in some database so you can later reference it.

    Hope this helps.

    • 0