Question

Here is the code for the relevant function:

def populateSubscribers(csvfile, db):
  conn = sqlite3.connect(db)
  conn.text_factory = str  #bugger 8-bit bytestrings
  cur = conn.cursor()

  subscriber_dict = {}

  # read values from tab-delimited csv file
  reader = csv.reader(open(csvfile, "rU"), delimiter = '\t')
  for Number, Name, Message, Datetime, Type in reader:
    if str(Number)[:1] == '1':
      tmpNumber = str(Number)[1:]
      Number = int(tmpNumber)

      # check to ensure name/number not null
      if Number and Name:
        # add unique subscribers to dictionary
        subscriber_dict[Number] = Name
      else:
        print 'Subscriber missing name or number'

  # insert unique subscribers into subscriber table
  for number, name in subscriber_dict.items():
    cur.execute('INSERT OR IGNORE INTO subscriber (name, phone_number) VALUES (?,?)', (name, number))
    conn.commit()

  cur.close()
  conn.close()
  print '...Successfully populated subcriber table.'

It is reading subscriber names and phone numbers from a csv file and then it is supposed to write an entry for each unique subscriber into the database. I wanted the phone number to be the key in key/value pairs since it is unique. But for some reason, it is not reading all of the numbers from the data, it is missing some subscribers. If I make the name the key it misses as would be anticipated (de-duplicates all the Unknowns), but the phone number as key is missing some numbers altogether. Any ideas on how to fix the logic here?

Answer 1

Looks to me like if str(Number)[:1] == '1': is probably filtering out some of your data.

Add an else to that and print out any it’s rejecting. Those are probably the ones which are going wrong.

Either way, pare down your input data and find which one’s aren’t being used. Without seeing the data and the alternative you’ve said which does work, it’s hard to pin down the exact cause here.