Question

I am working with Jama Matrix. I used it for LSI. It works all fine. However, when I pass a big matrix like, 8000×8000 it kills my whole system. I am simply calling SVD then reducing matrix size and adding up. Nothing else !

Any idea? How can I solve this problem?

core2du

Ram = 10GB

Java runtime setting

-Xmx5000M

There is no other program running while I execute Jama matrix code

Answer 1

I also use the Jama for SVD and have the same problem in solving big matrix.
In order to reduce memory overflow cases, I have tuned SingularValueDecomposition.java to compact one. The tuning is that in the matrix A it has so many 0 (zero) value so the compact for all of matrices used such as A, U, V, Work, etc only allows memory for available value which is more than 0.
Before you use the compact SVD, you should make A matrix file like
r /t c /t value /n
r /t c /t value /n
… ‘/t’ and ‘/n’ means tab and new line respectively.

examples, (0, 0, 0), (0, 1, 0.5), (0, 2, 0), (0, 3, 0.2), (1, 0, 1), (1, 1, 0), (1, 2, 0), (1, 3, 0.3) and the matrix size is 2*4 (R*C) then you just make file like
MATRIXSIZE /t 2 /t 4 /n
0 /t 1 /t 0.5 /n
0 /t 3 /t 0.2 /n
1 /t 0 /t 1 /n
1 /t 3 /t 0.3 /n

If you would like to use it, please give me your email address ([e-mail]).
I will give more details to use it on email.

I checked the result was right. However, I am not sure how much effective it is for the computer.
Anyway, it works and shows better even if not much.