Question

I’m using RSS library so i can parse Atom and RSS in Ruby and Rails and store it in a model.
I’ve looked at the standard RSS library, but is there one library that will auto-detect that there is a new rss feed so i can update my database ?

what are the best practice to trigger an instruction in order to store the new rss feed ?
should i use threads to handle that problem ?is it going to be slow?
thank you for your help

Answer 1

OK heres the deal.

1. If you want a real fast feed parser go for Feedzirra. Does not work on windows. http://github.com/pauldix/feedzirra

2. Autodiscovery?

   -Theres truffle-hog if you don’t want to do GET redirects. http://github.com/pauldix/truffle-hog

   -Theres feedbag if you want to do GET redirects to find feeds from given urls. This is slower though. http://github.com/damog/feedbag

3. Feedzirra is the best bet if you want to poll for new entries for your feed. But if you want a more non-polling solution to your problem then i would suggest going through the pubsubhubbub spec. Make sure while parsing your feeds they are pubsubhubbub enabled. Check for the link tag. If it points to pubsubhubbub.appspot.com or any other pubsub enabled hub then just subscribe to the feed by sending a subscription request to the hub. You can then define a endpoint in your app which will in turn receive updated entry pings for your feed subscription from the hub. Just read the raw POST data and store it in your database. Stats are that 95% of the blogger blogs are pubsub enabled. That is a lot of data in your hands already. 🙂

4. If you are polling for changes then you should check the last-modified or etag from the header rather than parse the entire feed again. Saves you from wasting resources. Feedzirra takes care of this for you.