Question

After implementing some of the solutions in my previous question, I’ve come up with the following solution:

reader = open('C://text.txt') 
writer = open('C://nona.txt', 'w')
counter = 1    
names, nums = [], []    
row = reader.read().split(' ')
x = len(row)/2
for (a, b) in [(c, d) for c, d in zip(row[:x], row[x:]) if d!='na']:
    print counter
    counter +=1
    names.append(a)
    nums.append(b)

writer.write(' '.join(names))
writer.write(' ')
writer.write(' '.join(nums))

This program works quite well for a smaller sample data set. However it freezes up when I use the full data set and causes python to crash. Any suggestions on how I can overcome this?

Answer 1

What you should do is break your file up into two separate files. Your logic should do something like this:

1. Open data file
2. open name file
3. read next data
4. is it name? see 5. Otherwise see 6
5. write name to name file, see 3
6. is it number or na? close name file and open number file
7. read next data
8. is it number or na? see 7, otherwise write file

once you have your files split into two pieces, you can iterate over them together:

names = open('names.txt')
numbers = open('numbers.txt')

for name, number in zip(names, numbers):
   if not numbers == 'na':
       output.write(name + " " + number)

or you could write to two different files and then join them together if that’s what you need.