Question

I created two functions: generating data and processing data. Data processing is time-consuming, so I want to process them in a parallel thread. But I have some problems with them. At first, here is my program:

result = zeros(1, 10);

matlabpool open local 2
spmd
    for a = 1:5
        data = generate_data();
        display(sprintf('Received data on CPU%d: %d', labindex, data));
        result(end + 1) = process_data(data);
    end
    display(sprintf('All done on CPU%d', labindex));
end
matlabpool close

And log of what it returned:

Starting matlabpool using the 'local' profile ... connected to 2 workers.
Lab 1: 
  Received data on CPU1: 100
Lab 2: 
  Received data on CPU2: 100
Lab 1: 
  Received data on CPU1: 101
  Received data on CPU1: 102
  Received data on CPU1: 103
  Received data on CPU1: 104
  All done on CPU1
Lab 2: 
  Received data on CPU2: 101
  Received data on CPU2: 102
  Received data on CPU2: 103
  Received data on CPU2: 104
  All done on CPU2
Sending a stop signal to all the workers ... stopped.

There is problem, which I have:

1. Values which it is returning by generate_data are the same for
   both thread. I should be different. Threads should process different
   data, rather than same data twice. I cannot generate the entire data
   set at once and use getLocalPart.

2. Variable result isn’t a 1×10 matrix of doubles, but 1×2 matrix of
   composites. I read about (co)distributed array but it didn’t help
   me. What I should do to receive the 1×10 matrix of doubles?

3. What I should do to CPU1 processes CPU2’s data, when finishes to
   process own data? Generally I don’t have any idea how to do this.

4. It is possible to remove “Lab 1:” and “Lab 2:”? They are messing
   my log 🙂

Taking into consideration above, log (for the larger data set) should look this way:

Starting matlabpool using the 'local' profile ... connected to 2 workers.
Received data on CPU1: 100
Received data on CPU2: 101
Received data on CPU1: 102
Received data on CPU1: 103
Received data on CPU1: 104
Received data on CPU1: 105
Received data on CPU2: 106
Received data on CPU1: 107
Received data on CPU1: 108
Received data on CPU2: 109
All done on CPU1
All done on CPU2
Sending a stop signal to all the workers ... stopped.

Answer 1

Why don’t you use the much simpler parfor? At the moment you’re running the loop on each workers, and I assume that you wanted to run the iterations of the loop in parallel.

nIter = 10;
result = zeros(1, nIter);

matlabpool open local 2

    parfor a = 1:nIter
        data = generate_data();
        fprintf('%s: processing set %i/%i\n',datestr(now),a,nIter)
        result(a) = process_data(data);
    end
end
matlabpool close