Question

I’m working on a site now that have to fetch users feeds. But how can I best optimize fetching if I have a database with, lets say, 300 feeds. I’m going to set up a cron-job to which fetches the feeds, but should I do it like 5 every second minute or something?

Any ideas on how to do this the best way in PHP?

Answer 1

Based on the new information I think I would do something like this:

Let the ‘first’ client initiate the updatework and store timestamp with it. Everey other clients that will ask for the information get a cashed information until that information are to old. Next hit from a client will then refresh the cashe that then will be used by all clients till next time its to old.

The client that will actually initiate the updatework should not have to wait for it to finnish, just serv the old cashed version and continue to do it till the work is done.

That way you dont have to update anything if no clients are requesting it.