Question

I have a small file read routine and I want only the 1st 200 records I have it working but along the way I could not figure out what was wrong with using the “while” construct.
This code works:

import csv, sys, zipfile
sys.argv[0] = "/home/tom/Documents/REdata/AllListing1RES.zip"
zip_file    = zipfile.ZipFile(sys.argv[0])
items_file  = zip_file.open('AllListing1RES.txt', 'rU')
rows = []
for row_index, row in enumerate(csv.DictReader(items_file, dialect='excel', delimiter='\t')):
    if (row_index < 200):
        rows.append(row)
    else : break

This code runs until it fails with an out of memory condition I would have thought it was equivalent?

import csv, sys, zipfile
sys.argv[0] = "/home/tom/Documents/REdata/AllListing1RES.zip"
zip_file    = zipfile.ZipFile(sys.argv[0])
items_file  = zip_file.open('AllListing1RES.txt', 'rU')
rows = []
for row_index, row in enumerate(csv.DictReader(items_file, dialect='excel', delimiter='\t')):
    while (row_index < 200):
        rows.append(row)
    else : break

so what would be the right construct using while? –

Answer 1

The more traditional way of writing that loop would be:

for row_index, row in enumerate(csv.DictReader(items_file, dialect='excel', delimiter='\t')):
    if (row_index >= 200):
        break
    rows.append(row)

As soon as the row counter hits 200, we bail out of the loop.

To use a while loop instead of a for loop (note that, as a looping construct, while is an alternative to for rather than to if) it is necessary to step through the iterator manually:

itr = enumerate(csv.DictReader(items_file, dialect='excel', delimiter='\t'))
row_index = -1
while row_index < 199:
    try:
        row_index, row = next(itr) # Python 3. Use itr.next() in Python 2
    except StopIteration:
        break # Ran out of data
    rows.append(row)

All that said, there’s actually a superior alternative to both of these options available in the itertools module:

from itertools import islice
itr = csv.DictReader(items_file, dialect='excel', delimiter='\t')
rows = list(islice(itr, 200))