Question

I’m running a for loop to grab content out of some XML and it works fine, until I reach around the 29th iteration. At that point it gives me this error:

File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2\webapp2.py", line 572, in dispatch
  return self.handle_exception(e, self.app.debug)   
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2\webapp2.py", line 570, in dispatch
  return method(*args, **kwargs)   
File "J:\Art & Graphic Design\Graphic Design\Websites\lawvoter-dev\cron_congressman.py", line 64, in get
  birthday      = re.findall("<birthday>(.*)</birthday>",element)[0] 
IndexError: list index out of range

The code is:

for element in members:
            title         = re.findall("<title>(.*)</title>",element)[0]
            role          = re.findall("<role_type_label>(.*)</role_type_label>",element)[0]
            name_sortable = re.findall("<name_sortable>(.*)</name_sortable>",element)[0]
            firstname     = re.findall("<firstname>(.*)</firstname>",element)[0]
            lastname      = re.findall("<lastname>(.*)</lastname>",element)[0]
            gender        = re.findall("<gender_label>(.*)</gender_label>",element)[0]
            birthday      = re.findall("<birthday>(.*)</birthday>",element)[0]
            party         = re.findall("<party>(.*)</party>",element)[0]
            state         = re.findall("<state>(.*)</state>",element)[0]
            description   = re.findall("<description>(.*)</description>",element)[0]
            start_date    = re.findall("<startdate>(.*)</startdate>",element)[0]
            end_date      = re.findall("<enddate>(.*)</enddate>",element)[0]
            website       = re.findall("<website>(.*)</website>",element)[0]
            bioguideid    = re.findall("<bioguideid>(.*)</bioguideid>",element)[0]
            osid          = re.findall("<osid>(.*)</osid>",element)[0]
            pvsid         = re.findall("<pvsid>(.*)</pvsid>",element)[0]
            twitterid     = re.findall("<twitterid>(.*)</twitterid>",element)[0]
            youtubeid     = re.findall("<youtubeid>(.*)</youtubeid>",element)[0]

            member = Congressman(title=title, role=role, name_sortable=name_sortable, firstname=firstname, lastname=lastname, gender=gender, birthday=birthday, party=party, state=state,
                                 description=description, start_date=start_date, end_date=end_date, website=website, bioguideid=bioguideid, osid=osid, pvsid=pvsid, twitterid=twitterid, youtubeid=youtubeid)
            member.put()

I really don’t get why this error is popping up? It always works fine for the first 29 iterations? The each element in the datamodel is also set to “default=None” just in case. However, when I look at the XML itself, and go to the exact line where the error happens, the value is actually there. Anyone know why it would give an error even though the value exists?

Answer 1

it looks like

birthday      = re.findall("<birthday>(.*)</birthday>",element)[0]

returns an empty list and you are trying to extract the first element which is not in the list and it throws the

IndexError: list index out of range

like here:

>>> l = []
>>> l[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> 

EDIT:

import re, logging

def findelement(item, element):
    i = re.findall(item, element)
    if not i:
        logging.info('no item found for %s with element %s' %(item, element))
        return ''
    return i[0]


for element in members:
    title = findelement("<title>(.*)</title>", element)
    ...