Question

I am designing, for my A2 Computing project, an application to simulate the response of a variety of filters. One problem I’m having is that the export data option is really slow.

Normally, when generating data to show on the display it’s about 40000 – 80000 points/sec. When recording it to a file, it drops to about a fifth of that.

Initially I thought my problem was because I was calling writeln on every data point. So I wrote it such that it queued the data up into a string and every 1000 points it wrote it in one big operation. It made it slightly faster, but still around 4-5x slower as displaying it on the built in form.

Why might this be?

Here’s the export code:

   for xx := 0 to npoints do
   begin
     freq := minfreq + ((xx / npoints) * maxfreq);
     ampl := GetAmplPoint(freq);
     phase := GetPhasePoint(freq);
     tempstr := tempstr + FormatFloat('#.#####', freq) + ',';
     tempstr := tempstr + FormatFloat('#.#####', ampl) + ',';
     tempstr := tempstr + FormatFloat('#.#####', phase) + sLineBreak;
     // Queue up to 1000 points, then write the data in one lump:
     // most of the time is spent in writeln waiting for IO which
     // slows down export.
     if xx mod 1000 = 0 then
     begin
       write(fileptr, tempstr);
       tempstr := '';
       ProgressBar.Position := 4 + Trunc((xx / npoints) * 96);
     end;
   end;

Answer 1

Disk I/O is one of the slowest bottlenecks today, especially if you use slow disks (i.e. the 4200/5400 rpm disks found on many laptops).

Better perfomance can be obtained using buffered I/O (old pascal I/O functions were designed long ago, and may use small buffers, better to use one of the buffered streams available today in Delphi) or asynch I/O (you pass the buffer to write to the OS, the call returns immediately and later the OS will tell you when it had written the data).