So, I’m finding the Eigen package crashes when I try to declare a matrix larger than 10000×10000. I need to declare a matrix like this.. about 13000×13000 elements reliably. I ran a test like:
for( int tortureEigen = 1 ; tortureEigen < 50000 ; tortureEigen++ )
{
printf( "Torturing Eigen with %dx%d..\n", tortureEigen, tortureEigen ) ;
Eigen::MatrixXd m( tortureEigen, tortureEigen ) ;
}
Crashes on my machine (6 GB RAM) at 14008 elements.
I’m kind of disappointed! I thought Eigen was like MATLAB or octave and should not crash using larger arrays, even if it does hit the disk or something..
And what’s more is when I run this test and keep TaskMan open, the process that is creating these matrices doesn’t even use that much memory. TaskMan reports under 2k usage.
Using Eigen 2.0.15 stable release
All the answers here are helpful!
It turns out that when compiled as a 32-bit app, Eigen will crash if you try and declare a dense MatrixXd, as I was, larger than 14000 elements or so. The crash happens with
_aligned_mallocreturning 0 in the Eigen alloc code (MatrixXd::resize()), meaning 1.5GB of contiguous, aligned RAM couldn’t be allocated under 32-bit, which makes sense, since this is getting close to half the maximum addressable memory loc. Finding more than 1.5 GB contiguous out of 4.0 becomes really unlikely, I suppose! Upgrading to Eigen 3.0 unfortunately does not solve the problem.Solution #1
Ok then, so I compiled in 64-bit, and on my 6GB machine, the program runs successfully, with the dense MatrixXd allocation and solution working just fine.
Solution #2
Another solution is using a
DynamicSparseMatrix<double>. Sparse does not crash on huge size alloc, even as 32 bit app, but API support for solving is another story (API seems to want to convert to MatrixXd dense type in order to solve, which leaves us with the same original problem.)