Question

I have a large loop where I am trying to calculate certain attributes for each pixel in a DEM (4800×6000). I am calling a function demPHV in which I’ve vectorized all calculations that outputs a structure with 26 fields . I have 4 cores but also have access to a multi-core cluster. I would like to speed up the weeks it would take to run this.

Z is the dem for this example. R is the spatialref object (a vector for example’s sake). latlim and lonlim are vectors of lat and long of the western US coastline (made up pairs in the example).
for example:

Z=rand(48,60);
R=makerefmat(120,40,.5,.5)
latlim=[40:60]';
lonlim=[136:(143-136)/(length(latlim)-1):143]';

Then my original loop:

for col=11:size(Z,2)-11
    for row=11:size(Z,1)-11
        dpv=demPHV(Z,R,row,col,latlim,lonlim)

    fn=fieldnames(dpv);
    for k=1:length(fieldnames(dpv))
        DEM_PHV.(fn{k}).{row,col}=dpv.(fn{k});
    end
end

Loops for parallelizing:

option 1:

[rows, cols] = meshgrid(12:(size(Z,1)-12), 12:(size(Z,2)-12));
inds = sub2ind(size(Z), rows, cols);
inds = inds(:)';
parfor i=inds(1):inds(end)
       dpv=demPHV(Z,R,i,latlim,lonlim)
end

This includes [r,c]=ind2sub(size(Z),i) in the function to use in the function demPHV.

option 2:

parfor col=11:size(Z,2)-11
    for row=11:size(Z,1)-11
         dpv=demPHV(Z,R,row,col,latlim,lonlim)
    end
end

parfor requires consecutive integers hence some of these changes. I have to exclude the bordering 11 rows and columns because my function uses surrounding pixels to calculate some of the attributes.

So, my questions:

1. Would you expect either of these two options to be faster than the other?

2. parfor does not allow me to include the second part of my original loop:

   fn=fieldnames(dpv);

   for k=1:length(fieldnames(dpv))

   DEM_PHV.(fn{k}).{row,col}=dpv.(fn{k});

   end

during which I assign the output structure to another variable. The ultimate goal is to have the variable DEM_PHV have fields for every attribute I need, and every field to be a matrix size(Z) where every cell is the corresponding value for that attribute. I’ve tried to have my function output the values in the correct cell of the matrix, but then I get a matrix size(Z) with [] everywhere except for the value at location row,col. This seems like a horribly inefficient use of memory… any better suggestions? I hope I covered everything.
Thanks for looking!

Answer 1

I got the following code to work and store each of the function results into an array of structures. There might be a better way though, because now I need to extract each of the value from each of the fields from the structures into its own matrix.

Does anyone have a suggestion for creating something like: DEM_PHV = struct('field1',[dPHV{:}.field1],'field2',[dPHV{:}.field2]) where each matrix in each field is size(Z). The cells in the matrix will contain either a single value or a pair such as [lat, long].

dPHV=cell(110,110);
parfor col=11:110%size(Z,2)-11
    for row = 11:110%size(Z,1)-11
       pixel_attributes=demPHV(Z,R,row,col,latlim,lonlim); %function produces structure of variables, each iteration is another pixel
       dPHV{row,col}=structfun(@(x) x,pixel_attributes,'UniformOutput',false)   
    end
end  

EDIT: as kludgey as this may be, the following works for reassigning the variables. I would gladly take suggestions for a slicker ‘MATLAB’ way.

% find all field names and get size of output
fn=fieldnames(dPHV{11,11});
[I, J]=size(dPHV);

%initialize final output
for f=1:numel(fn)
    DEM_PHV.(fn{f})=cell(size(dPHV));
end
% loops through datastructure to populate new fields
for i=11:I-11
    for j=11:J-11
        for f=1:numel(fn)
            DEM_PHV.(fn{f}){i,j}=dPHV{i,j}.(fn{f});
        end
    end
end