Home/ Questions/Q 747999
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-14T14:16:10+00:00

I’m building a database + tool that scours news feeds for a certain term.

  • 0

I’m building a database + tool that scours news feeds for a certain term. For example “food poisoning from nuts”. I want to scour social media sites, news sites, major news aggregators, etc… for that term.

  • Question 1: What are some of the news
    aggregator APIs out there?
  • Question 2: How Would you go about coding and
    receiving only the latest news from
    the API?

Edit Added schematic:
alt text http://koopics.com/news_parser.jpg

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Do you know Yahoo! Pipes? It’s a very flexible feed aggregator, and you can manipulate it using YQL, which is quite powerful and has a Python librabry, python-yql :). YQL also has specific “tables” for Twitter and other services and news sources, so, depending on what you want, you might not even need Pipes.

    There’s a quick example of Pipes + YQL usage at http://blog.ouseful.info/2009/04/27/using-yql-with-yahoo-pipes/ , but you can play around with them at the YQL Console.

    So, that’s your “Question 1”. But, using YQL, “Question 2” (if I understood it correctly) comes naturally, as you use it almost like regular SQL, imposing limits, ordering etc.

    • 0