Question

I need to concatenate multiple files’ data into one matrix.
So far, the way that I have been testing loading my data is something akin to the following:

fid = fopen('data01.txt', 'r');
raw = textscan(fid, '%d/%d/%d %d:%d:%f %f %f %f %d', 'delimiter', ',');
m = cellfun(@double, raw, 'UniformOutput', false);
value_of_interest = m{:,10}

…But the data set that I have on disk is many files and all exist within a single directory. I’d prefer to refer to a specific path for this directory, rather than placing my script there. How can I modify my script so that it loads all data for all of the files in said folder?

So far I have this:

dirname = uigetdir;
files = dir(dirname);
fileIndex = find(~[files.isdir]);

for i = 1:length(fileIndex)
    fileName = files(fileIndex(i)).name;
    fid = fopen(fileName, 'r');
    raw = textscan(fid, '%d/%d/%d %d:%d:%f %f %f %f %d', 'delimiter', ',');
    time = [m{:,4}, m{:,5}, m{:,6}];  %needs to contain a float
    converted_time = ((m{:,4} *  3600.0) + (m{:,5} * 60.0) + m{:,6}); %hh:mm:ss -> seconds
    values = power(m{:,10}, 2); 
    values(values <= thresh) = 0;
    % need to concat into the var 'values' here... also need to accumulate the time variable
end 

plot(converted_time, values);

…But I need to put the two together.

EDIT: I should mention that I may run out of memory, which is explained later in comments below to my chosen answer.

Answer 1

First, have another look at how you are defining the fileName of the file to be opened. Instead, you should try fileName = [dirname, '\', files(fileIndex(i)).name];, since the name field of files will not contain the full path. This will solve your problem of referencing a list of files that are not in your current path.

Now, to avoid remembering all the data from all those files, we can do this job per file inside the loop:

...
    plot(converted_time, values);
    hold('on');
end

The short command hold('on');, often written simply hold on; modifies the plot axes such that subsequent data can be plotted without erasing any previous lines.