Question

I have a large matrix (2e6 x 3) which I have to write to a text file.

dlmwrite takes about 230s to achieve this task.

From your experience what is the fastest way to write a large matrix to a text file?

Answer 1

The following applies to MATLAB, but I suggest you try it in Octave. First of all, if you can – transpose the matrix. Here are examples using fprintf and csvwrite (essentially dlmwrite)

A = rand(3, 1e6);
tic;
fid = fopen('data.txt', 'w+');
for i=1:size(A, 1)
    fprintf(fid, '%f ', A(i,:));
    fprintf(fid, '\n');
end
fclose(fid);
toc

tic;
csvwrite('data.txt', A);
toc;

Elapsed time is 1.311512 seconds.
Elapsed time is 2.487737 seconds.

If not transposed, it will take ages indeed. By default, fprintf flushes the buffer after every call. You can try to use W instead of w to open the file, but it does not improve the situation here too much.