Question

The code below is streaming the twitter public timeline for a variable which output any tweets to the console. I’d like the save the same variables (status.text, status.author.screen_name, status.created_at, status.source) into an sqlite database. I’m getting an syntax error when my script sees a tweet and nothing is written to the sqlite database.

the error:

$ python stream-v5.py @lunchboxhq
Filtering the public timeline for "@lunchboxhq"RT @LunchboxHQ: test 2   LunchboxHQ  2012-02-29 18:03:42 Echofon
Encountered Exception: near "?": syntax error

the code:

import sys
import tweepy
import webbrowser
import sqlite3 as lite

# Query terms

Q = sys.argv[1:]

sqlite3file='/var/www/twitter.lbox.com/html/stream5_log.sqlite'

CONSUMER_KEY = ''
CONSUMER_SECRET = ''
ACCESS_TOKEN = ''
ACCESS_TOKEN_SECRET = ''

auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)

con = lite.connect(sqlite3file)
cur = con.cursor()
cur.execute("CREATE TABLE TWEETS(txt text, author text, created int, source text)")

class CustomStreamListener(tweepy.StreamListener):

    def on_status(self, status):

        try:
            print "%s\t%s\t%s\t%s" % (status.text, 
                                      status.author.screen_name, 
                                      status.created_at, 
                                      status.source,)

            cur.executemany("INSERT INTO TWEETS(?, ?, ?)", (status.text, 
                                                            status.author.screen_name, 
                                                            status.created_at, 
                                                            status.source))

        except Exception, e:
            print >> sys.stderr, 'Encountered Exception:', e
            pass

    def on_error(self, status_code):
        print >> sys.stderr, 'Encountered error with status code:', status_code
        return True # Don't kill the stream

    def on_timeout(self):
        print >> sys.stderr, 'Timeout...'
        return True # Don't kill the stream

streaming_api = tweepy.streaming.Stream(auth, CustomStreamListener(), timeout=60)

print >> sys.stderr, 'Filtering the public timeline for "%s"' % (' '.join(sys.argv[1:]),)

streaming_api.filter(follow=None, track=Q)

Answer 1

You are missing a closing parenthesis on the last line of the following code (lines 34–37 from what you posted):

            cur.executemany("INSERT INTO TWEETS(?, ?, ?)", (status.text, 
                                                        status.author.screen_name, 
                                                        status.created_at, 
                                                        status.source)

Just add a parenthesis to close the method call immediately after your tuple parameter.