Question

I know long titel — I could not think of anything else 😉

So I am writing a python script which will save elements of the Twitter Search API to a csv file

writer = csv.writer(open('stocks.csv', 'a', buffering=0))
writer.writerows([(screen_name, hashtags, expanded_url , coordinates , geo , in_reply_to_user_id, followers)])

But I want to add how many followers the tweeting user has!

Now this is done via the GET users/lookup Twitter API which is limited to 350 requests per hour but allows simultaneous look up of up to 100 users

right now my script when finding a tweet looks up the users followers and pasts it with all the info of the tweet into the csv file.

This works great but after 350 searches I hit my limit!!!!

Now my question is:
Can I make the script search 100 times and store the hundred usernames somewhere and once it hits 100 it calles the GET users/lookup and inserts the info right of the search info into the excel file:

Excel example:

 [info from search ...(in many columns)] [followers of the user who sent the tweet]
 [info from search ...(in many columns)] [followers of the user who sent the tweet]
 [info from search ...(in many columns)] [followers of the user who sent the tweet]

As per request:

import urllib2
import urllib
import json
import time


 s = u'@apple OR @iphone OR @aapl OR @imac OR @ipad OR @mac OR @macbook OR macbook OR mac OR ipad OR iphone 4s OR iphone 5 OR @iphone4s OR @ iphone 5 OR aapl OR iphone'

info =  urllib2.quote(s.encode("utf8"))
page = "?q="

 openurl = urllib.urlopen("http://search.twitter.com/search.json"+ page + info)

quota = 150
user = 'twitter'
user_info = urllib.urlopen("https://api.twitter.com/1/users/lookup.json?screen_name="+user)

while quota > 10:
 openurl2 = urllib.urlopen("https://api.twitter.com/1/account/rate_limit_status.json")
 twitter_quota = openurl2.read()
 quota_json = json.loads(twitter_quota)
 quota = quota_json['remaining_hits']


 twitter_search = openurl.read()

 table_search = json.loads(twitter_search)
 print table_search

 print str(table_search[u'results'][1][u'iso_language_code'])


 lines = 0

 linesmax = len(table_search[u'results'])
 print linesmax

 while lines < linesmax:
    table_timeline_inner = table_search[u'results'][lines]


    next = table_search[u'next_page']
    lang = table_timeline_inner[u'iso_language_code']
    to = table_timeline_inner[u'to_user_name']
    text = table_timeline_inner[u'text']
    user = table_timeline_inner[u'from_user']
    geo = table_timeline_inner[u'geo']
    time = table_timeline_inner[u'created_at']
    result_type = table_timeline_inner[u'metadata'][u'result_type']
    id = table_timeline_inner[u'id']

Answer 1

Yes. Instead of writing each row as soon as you get it, store it in a temporary set. When this set has 100 elements, look them all up using one request. Then iterate over each element of the set and write it together with the data you got from the request, which you will have to match up by userid or something.

_{I’m not going to write the code for you…}