The meetup API allows you to get pushed data updates. A CURL request that looks as followed:
http://stream.meetup.com/2/open_events
Will return a constant stream of updated events. This imported stream just keeps updating as new data is pushed to it. I am trying to think up the best way to work with this stream, and I am not sure what is standard. I was thinking to just run a CURL request within a while loop but that would just run forever and running that from a controller function in my codeigniter project seems impractical. What is the proper way to interface with a seemingly infinite stream of events like this? Is there a better way than running a process from a controller function? Would it be more sensible to run a chron-job that woke up every so often to continue processing from where it left off before?
UPDATE:
Further investigation has shown me that importing data from this stream uses chunked-transfer encoding. What is confusing to me is that this stream seems to be infinite. It just keeps outputting data non-stop. I still do not understand how to import data from a stream of data like this. A cron job also does not seem to be the answer for importing a stream like this into my mysql database.
This is a link to the resource that documents the Meetup Stream:
http://www.meetup.com/meetup_api/docs/stream/2/open_events/
Thank you for the help.
Without looking closely at the meetup.com docs, it sounds like you would probably want to use javascript & ajax to call your controller (which calls the meetup web service) at fixed intervals. Your Ajax response would then contain your updated data, and you could update your page accordingly in the ajax callback function.
Check out the javascript setInterval() method