I wrote a program using Tweepy that I can run on my home computer to monitor a Twitter stream. The way Tweepy does this is basically to have a while(true) loop always running. Now what I want to do is link this program into my Django app. One way I could do this is to leave the Twitter program on the local computer and have it update the database of the server when it receives a message. However, I was wondering what I need to do run the Twitter program in the background on my server. I am using Django.
Share
As you have discovered with your comment, celery is not ideal – you would need another long running process to monitor the existing long running process (your client).
Ideally you need a socket that is always open (like the infinite while loop in your client); so wheverver there is data you get a “realtime” view of the twitter feed on the web page.
Node.js combined with SocketIO is designed to solve this, and since a twitter feed realtime viewer is a common case many examples are floating about – streamie is one of them.
That combination gives you a javascript client that automatically updates based on data coming; no refresh, ajax polling, cron, etc. required.