Question

There are some questions discuss about how to use Twitter API to get tweets from a user. (e.g., Twitter API getting any user's all tweets, Is there a way to get all tweets from twitter for a specified user?)

I know I should use user_timeline. But my questions is that Twitter put a rate limit, if I want to download say 1000 tweets a time, it will give me an error.

In the following code I made separate requests to the API and sleep several minutes between each request, but it still doesn’t work.

for p in range(1, 6):
    params['page'] = p
    url = base_url + urllib.urlencode(params)

    r = requests.get(url)
    j = simplejson.loads(r.content)

    for item in j:
       # do something to tweets

time.sleep(180)

Any idea for work around? Thanks.

Answer 1

You can access the streaming API using e.g. Tweepy, and filter the API statuses method with one or more user IDs. The advantage to this method is that the streaming API isn’t rate-limited. This answer should get you up and running with Tweepy and the streaming API.