Question

Have two scripts that give very different results. First using csv.reader and it works fine, then csv.dictreader which has given me problems. Both scripts use the same set of data, only difference between the two files is that the one dictreader uses has headers.

import csv

inv = csv.reader(open('inv.txt', 'rU'), dialect='excel', delimiter="\t")

for PART_CODE,MODEL_NUMBER,PRODUCT_NAME,COLOR,TOTAL_ONHAND,TOTAL_ON_ORDER,TOTAL_SALES,SALES_YEAR_TO_DATE,SALES_LASTYEAR_TO_DATE,\
TOTAL_NUMBER_OF_QTYsSOLD,TOTAL_PURCHASES,PURCHASES_YEAR_TO_DATE,PURCHASES_LASTYEAR_TO_DATE,TOTAL_NUMBER_OF_QTYpurchased,DATE_LAST_SOLD,DATE_FIRST_SOLD in inv:
    if int(TOTAL_ON_ORDER) >= 1:
        print ('%-20s %-100s OnHand: %-4s OnOrder: %-4s') % (MODEL_NUMBER,PRODUCT_NAME,TOTAL_ONHAND,TOTAL_ON_ORDER)

The above works just fine, it’ll parse through the 20,000 plus items with no error. Now if I choose to use dictreader as in the below, the script will run into problems after a while…

import csv

inv = csv.DictReader(open('ireport.txt', 'rU'), dialect='excel', delimiter="\t")

for row in inv:
    if int(row['TOTAL_ON_ORDER']) >= 1:
        print ('%-20s %-100s OnHand: %-4s OnOrder: %-4s') % (row['MODEL_NUMBER'],row['PRODUCT_NAME'],row['TOTAL_ONHAND'],row['TOTAL_ON_ORDER'])

Prints out about 100 or so, then fails and reports this error:

if int(row['TOTAL_ON_ORDER']) >= 1:

ValueError: invalid literal for int() with base 10: 'False'

Puzzles me as both scripts use the same data (other than the reader one has no header row and the dictreader one does) one works flawlessly, the other complains. Any clues?

Snippet of inv.txt:

61965901576 383964  Sandisk 128MB 3.3V Smartmedia Card      0   0   0   0   0   0   0   0   0   0   00/00/00    00/00/00
61965901521 348236  Sandisk 128MB Compactflash Card     0   0   54.26   0   0   1   0   0   0   0   01/09/02    01/09/02
61965902011 SDCZ2-1024-A10  Sandisk 1GB Cruzer Mini USB Flash Drive     0   0   0   0   0   0   0   0   0   0   00/00/00    00/00/00
61965901571 266796  Sandisk 256MB CompactFlash Disk     0   0   678.22  0   0   5   0   0   0   0   06/27/02    03/08/02

Answer 1

It looks like it’s reading something as a Boolean into an integer (which it’s not happy with) in the DictReader function, whereas in the reader function it is not getting that cast as such.

Try this:

import csv

inv = csv.DictReader(open('ireport.txt', 'rU'), dialect='excel', delimiter="\t")

for row in inv:
    try:
      if int(row['TOTAL_ON_ORDER']) >= 1:
          print ('%-20s %-100s OnHand: %-4s OnOrder: %-4s') % (row['MODEL_NUMBER'],row['PRODUCT_NAME'],row['TOTAL_ONHAND'],row['TOTAL_ON_ORDER'])
    except Exception as Err:
      print row['TOTAL_ON_ORDER'],Err
      break #if you want to end the function)

This will show you what line it’s choking on, and if you remove the break should chug through it.

Good luck!