Question

I’m parsing a very big csv (big = tens of gigabytes) file in python and I need only the value of the first column of every line. I wrote this code, wondering if there is a better way to do it:

delimiter = ','
f = open('big.csv','r')
for line in f:
    pos = line.find(delimiter)
    id = int(line[0:pos])

Is there a more effective way to get the part of the string before the first delimiter?

Edit: I do know about the CSV module (and I have used it occasionally), but I do not need to load in memory every line of this file – I need the first column. So lets focus on string parsing.

Answer 1

>>> a = '123456'
>>> print a.split('2', 1)[0]
1
>>> print a.split('4', 1)[0]
123
>>> 

But, if you’re dealing with a CSV file, then:

import csv
with open('some.csv') as fin:
    for row in csv.reader(fin):
        print int(row[0])

And the csv module will handle quoted columns containing quotes etc…