Question

I am learning Python (I have a C/C++ background).

I need to write something practical in Python though, whilst learning. I have the following pseudocode (my first attempt at writing a Python script, since reading about Python yesterday). Hopefully, the snippet details the logic of what I want to do. BTW I am using python 2.6 on Ubuntu Karmic.

Assume the script is invoked as: script_name.py directory_path

import csv, sys, os, glob

# Can I declare that the function accepts a dictionary as first arg?
def getItemValue(item, key, defval)
  return !item.haskey(key) ? defval : item[key]


dirname = sys.argv[1]

# declare some default values here
weight, is_male, default_city_id = 100, true, 1 

# fetch some data from a database table into a nested dictionary, indexed by a string
curr_dict = load_dict_from_db('foo')

#iterate through all the files matching *.csv in the specified folder
for infile in glob.glob( os.path.join(dirname, '*.csv') ):
  #get the file name (without the '.csv' extension)
  code = infile[0:-4]
  # open file, and iterate through the rows of the current file (a CSV file)
  f = open(infile, 'rt')
  try:
    reader = csv.reader(f)
    for row in reader:
      #lookup the id for the code in the dictionary
      id = curr_dict[code]['id']
      name = row['name']
      address1 = row['address1']
      address2 = row['address2']
      city_id = getItemValue(row, 'city_id', default_city_id)

      # insert row to database table

  finally:
    f.close()

I have the following questions:

1. Is the code written in a Pythonic enough way (is there a better way of implementing it)?

2. Given a table with a schema like shown below, how may I write a Python function that fetches data from the table and returns is in a dictionary indexed by string (name).

3. How can I insert the row data into the table (actually I would like to use a transaction if possible, and commit just before the file is closed)

Table schema:

create table demo (id int, name varchar(32), weight float, city_id int);

BTW, my backend database is postgreSQL

[Edit]

Wayne et al:

To clarify, what I want is a set of rows. Each row can be indexed by a key (so that means the rows container is a dictionary (right)?. Ok, now once we have retrieved a row by using the key, I also want to be able to access the ‘columns’ in the row – meaning that the row data itself is a dictionary. I dont know if Python supports multidimensional array syntax when dealing with dictionaries – but the following statement will help explain how I intend to conceptually use the data returned from the db. A statement like dataset[‘joe’][‘weight’] will first fetch the row data indexed by the key ‘joe’ (which is a dictionary) and then index that dictionary for the key ‘weight’. I want to know how to build such a dictionary of dictionaries from the retrieved data in a Pythonic way like you did before.

A simplistic way would be to write something like:

import pyodbc

mydict = {}
cnxn = pyodbc.connect(params)
cursor = cnxn.cursor()
cursor.execute("select user_id, user_name from users"):

for row in cursor:
   mydict[row.id] = row

Is this correct/can it be written in a more pythonic way?

Answer 1

to get the value from the dictionary you need to use .get method of the dict:

>>> d = {1: 2}
>>> d.get(1, 3)
2
>>> d.get(5, 3)
3

This will remove the need for getItemValue function. I wont’ comment on the existing syntax since it’s clearly alien to Python. Correct syntax for the ternary in Python is:

true_val if true_false_check else false_val
>>> 'a' if False else 'b'
'b'

But as I’m saying below, you don’t need it at all.

If you’re using Python > 2.6, you should use with statement over the try-finally:

with open(infile) as f:
    reader = csv.reader(f)
    ... etc

Seeing that you want to have row as dictionary, you should be using csv.DictReader and not a simple csv. reader. However, it is unnecessary in your case. Your sql query could just be constructed to access the fields of the row dict. In this case you wouldn’t need to create separate items city_id, name, etc. To add default city_id to row if it doesn’t exist, you could use .setdefault method:

>>> d
{1: 2}
>>> d.setdefault(1, 3)
2
>>> d
{1: 2}
>>> d.setdefault(3, 3)
3
>>> d
{1: 2, 3: 3}

and for id, simply row[id] = curr_dict[code]['id']

When slicing, you could skip 0:

>>> 'abc.txt'[:-4]
'abc'

Generally, Python’s library provide a fetchone, fetchmany, fetchall methods on cursor, which return Row object, that might support dict-like access or return a simple tuple. It will depend on the particular module you’re using.