Question

Basically, I’m trying to use the api offered by govtrack.us to pull information and store into my own app’s datastore for further manipulation (using python). The api serves json by default and when i get the json I start to get lost on how to add each element I want to the datastore. For example, this json has the following keys: {u’meta’, u’objects’}. Each object has the following keys:

[u'gender_label', u'osid', u'id', u'pvsid', u'current_role', u'name_sortable', u'firstname', u'twitterid', u'middlename', u'lastname', u'bioguideid', u'birthday', u'link', u'youtubeid', u'nickname', u'name', u'roles', u'gender', u'namemod', u'metavidid', u'name_no_details', u'resource_uri']

I want to be able to take each object and store it into the datastore (but not all the information just some – like u’current_role’, u’youtubeid’, u’name’, etc…).

Right now, I’ve got this function that pulls the json:

def get_congressman():
url = 'http://www.govtrack.us/api/v1/person?roles__current=true&limit=3000'
content  = None
try:
    content = urllib2.urlopen(url).read()
except URLError:
    return
if content:
    return content

And this to iterate over the returned json:

current_congressman = get_congressman()
j            = json.loads(current_congressman)
name         = [c['name_no_details'] for c in j['objects']]
youtube      = [c['youtubeid'] for c in j['objects']]
gender_list  = [c['gender_label'] for c in j['objects']]

Instead of adding all the people’s names, gender, youtube feeds, etc. to a seperate list, I would like to add each object to their own list containing the information needed to add to the datastore. Basically, a list like:

["Gary Ackerman", "Male", "RepAckerman"]

But one for each object. What would be the best way to go about this? Or do I have to have a list of all names, another list of all genders, and so forth and then match them up?

Answer 1

First you need to define a model. See the Ndb Overview, for details.

Lets say your model is something like

class Congressman(ndb.model):
    ...

Then instead of computing one list of names, other for youtube ids, you will traverse all the objects once and create a Congressman object and store it.

for congressman_info in j['objects']:
    congressman = Congressman(gender=congresmman_info['gender_label'],
                              name=congressman_info['name_no_details'], ...)
    congressman.put()