Question

This may seem like the worlds simplest python question… But I’m going to give it a go of explaining it.

Basically I have to loop through pages of json results from a query.

the standard result is this

{'result': [{result 1}, {result 2}], 'next_page': '2'}

I need the loop to continue to loop, appending the list in the result key to a var that can be later accessed and counted the amount of results within the list. However I require it to loop only while next_page exists as after a while when there are no more pages the next_page key is dropped from the dict.

currently i have this

next_page = True
while next_page == True:
    try:
        next_page_result = get_results['next_page'] # this gets the next page
        next_url = urllib2.urlopen("http://search.twitter.com/search.json" + next_page_result)# this opens the next page
        json_loop = simplejson.load(next_url) # this puts the results into json
        new_result = result.append(json_loop['results']) # this grabs the result and "should" put it into the list
    except KeyError:
        next_page = False   
        result_count = len(new_result)

Answer 1

Alternate (cleaner) approach, making one big list:

results = []
res = { "next_page": "magic_token_to_get_first_page" }
while "next_page" in res:
    fp = urllib2.urlopen("http://search.twitter.com/search.json" + res["next_page"])
    res = simplejson.load(fp)
    fp.close()
    results.extend(res["results"])