Question

According to this question, I should try to use Preallocation is Matlab.

Now I have a situation that I cannot calculate the exact size of the matrix to preallocate. I can guess the size.

suppose the actual size of the matrix is 100, but I don’t know it. Sh

Which scenario is more efficient:

1. Should I be lavish? I guess a large matrix and at the end I remove extra rows.
2. Should I be stingy? I guess a small size and If it was wrong, I add new rows.

Thanks.

Answer 1

To my opinion, the answer is a bit more complex than portrayed by @natan.
I think there are two factors his answer does not take into account:

1. Possible necessary copies of memory: when you under-estimate a matrix size and you re-allocate it, all its old values should be copied to the new allocated location.

2. Continuity of memory chunks: sometimes Matlab is able to allocate new memory continuously at the end of the old matrix. In principle, in such a scenario the old values need not be copied to the new location – since it is the same as the old one just bigger. However, if you add rows to a 2D matrix, the content needs to be copied even in this scenario, since Matlab stores matrices in a row-major fashion in memory.

So, my answer is this:

First of all, what exactly don’t you know about the size of the matrix: if you know one dimension – make it the number of rows of your matrix, so you’ll only need to change the number of columns. This way, if your already stored data needs to be copied, it would be copied at larger chunks.

Second, it depends on how much free RAM you have at your disposal.
If you are not short at RAM, then there’s nothing wrong with over estimating.

However, if you are short at RAM, consider under estimating. BUT when you re-allocate, increase the new block size at each iteration:

BASIC_SIZE = X;  % first estimate
NEW_SIZE = Y;    % if need more, add this amount
factor = 2;      
arr = zeros( m, BASIC_SIZE ); % first allocation, assuming we know number of rows
while someCondition
    % process arr ...
    if needMoreCols
        arr(:, size(arr,2) + (1:NEW_SIZE) ) = 0; % allocate another block
        NEW_SIZE = round(NEW_SIZE * factor);  % it seems like we are off in estimation, try larger chunk next time factor should be > 1
    end
end
arr = arr(:, 1:actualNumOfCols ); % resize to actual size, discard unnecessary columns