Question

I have a line of code in Matlab that reads:

  output = find(input);

where column vector “output” contains all indices in column vector “input” whose elements are nonzero. For example, if:

 input = [1 3 4 0 0 2 0];

then the result of, output = find(input); would be:

 output = 
     1
     2
     3
     6

corresponding to the 1st (“1”), 2nd (“3”), 3rd (“4”), and 6th (“2”) indices of array “input” that are nonzero.

Since my “input” array is very large, this line of code consumes all of my local RAM plus a huge portion of virtual memory, causing the system to slow to a crawl.

Anyone know of a good way (or any way) to reduce the memory requirements of such an operation? I thought about putting the “find” code in a loop, but since the size of array “output” (and thus indexing of this array) depends on the result of the “find” operation, I don’t see how it’s possible to do so. Ran out of ideas.

Thanks in advance for any comments/suggestions. -gkk

Answer 1

If you have enough RAM to hold an array the same size of input, you can replace the call to find by

output = 1:length(input);
output = output(input~=0);

If input has less than 2^32-1 elements, you can initialize it as uint32, and thus further save on memory.

A better way might be to convert your input array to sparse, which saves memory if input contains lots of zeros, and then use find on that, i.e.

input = sparse(input);
output = find(input);

---

EDIT

To perform the find operation in pieces, I’d do the following:

nIn = length(input);
blockLength = 100000;
nBlocks = ceil(nIn/blockLength); %# work in chunks of 100k entries
out = cell(nBlocks,1);
for i=1:nBlocks
   out{i} = (i-1)*blockLength+1:i*blockLength; %# assign as your favorite integer format here
   out{i} = out{i}(input(out{i})~=0);
end
out = cat(1,out{:});