Question

I’m aiming to import data from various folder and text files into matlab.

clear all
main_folder = 'E:\data';
    %Directory of data
TopFolder = dir(main_folder);
    %exclude the first two cells as they are just pointers. 
TopFolder = TopFolder(3:end);
TopFolder = struct2cell(TopFolder);
Name1 = TopFolder(1,:);
    %obtain the name of each folder
dirListing = cellfun(@(x)dir(fullfile(main_folder,x,'*.txt')),Name1,'un',0);
Variables = cellfun(@(x)struct2cell(x),dirListing,'un',0);
FilesToRead = cellfun(@(x)x(1,:),Variables,'un',0);
    %obtain the name of each text file in each folder

This provides the name for each text file in each folder within ‘main_folder’. I am now trying to load the data without using a for loop (I realise that for loops are sometimes faster in doing this but I’m aiming for a compact script).

The method I would use with a for loop would be:

for k = 1:length(FilesToRead);
    filename{k} = cellfun(@(x)fullfile(main_folder,Name{k},x),FilesToRead{k},'un',0);
    fid{k} = cellfun(@(x)fopen(x),filename{k},'un',0);
    C{k} = cellfun(@(x)textscan(x,'%f'),fid{k},'un',0);
end

Is there a method which would involve not using loops at all? something like cellfun within cellfun maybe?

Answer 1

folder = 'E:\data';
files = dir(fullfile(folder, '*.txt'));
full_names = strcat(folder, filesep, {files.name});
fids = cellfun(@(x) fopen(x, 'r'), full_names);
c = arrayfun(@(x) textscan(x, '%f'), fids);  % load data here
res = arrayfun(@(x) fclose(x), fids);
assert(all(res == 0), 'error in closing files');

but if the data is in csv format it can be even easier:

folder = 'E:\data';
files = dir(fullfile(folder, '*.txt'));
full_names = strcat(folder, filesep, {files.name});
c = cellfun(@(x) csvread(x), full_names,  'UniformOutput', false);

now all the data is stored in c