Question

I’m new to Python and I have a solution for this but it seems slow and silly, so I wondered if there is a better way?

Say I have a matrix defined like this:

mat = [['hello']*4 for x in xrange(3)]

I am using this function to write it to file:

def writeMat(mat, outfile):
  with open(outfile, "w") as f:
    for item in mat:
      f.writelines(str(item).replace('[','').replace(',','').replace('\'','').replace(']','\n'))

writeMat(mat, "temp.txt")

which gives a text file that looks like:

hello hello hello hello
hello hello hello hello
hello hello hello hello

The files that I am dealing with are very large. The savetxt function in numpy would be great, but I don’t want to store this as a numpy array because while the majority of the matrix is comprised of single character elements, the first few columns will be many characters in length, and it seems to me (correct me if I am wrong) this would mean the whole matrix would use much more memory than is necessary because every element in the matrix will be the size of the largest element.

Answer 1

If I understand your question correctly, you could do:

f.writelines(' '.join(row) + '\n' for row in mat)

or

f.write('\n'.join(' '.join(row) for row in mat))

The first one has the advantage of being a generator expression that only makes a concatenated string copy of the currentline

And if your matrix entries are not strings, you could do:

f.writelines(' '.join(str(elem) for elem in row) + '\n' for row in mat)

EDIT

It appears that the file.writelines() method evaluates the entire generator expression before writing it to the file. So the following would minimize your memory consumption:

for row in mat:
    f.write(' '.join(row) + '\n')