I’m trying to apply a Kalman Filter to sensor readings using Java, but the matrix manipulation library I’m using is giving me a heapspace error. So, does anyone know of a matrix manipulation library for the JVM with better memory allocation characteristics?
It would seem that this one — http://code.google.com/p/efficient-java-matrix-library/ — is “efficient” only in name. The data set has 9424 rows by 2 columns, all values are doubles (timestamp and one dimension out of 3 on readings from a sensor).
Many thanks, guys!
1) The Kalman filter should not require massive, non linear scaling amounts of memory : it is only calculating the estimates based on 2 values – the initial value, and the previous value. Thus, you should expect that the amount of memory you will need should be proportional to the total amount of data points. See : http://rsbweb.nih.gov/ij/plugins/kalman.html
2) Switching over to floats will 1/2 the memory required for your calculation . That will probably be insignificant in your case – I assume that if the data set is crashing due to memory, you are running your JVM with a very small amount of memory or you have a massive data set.
3) If you really have a large data set ( > 1G ) and halving it is important, the library you mentioned can be refactored to only use floats.
4) For a comparison of java matrix libraries, you can checkout http://code.google.com/p/java-matrix-benchmark/wiki/MemoryResults_2012_02 — the lowest memory footprint libs are ojAlgo, EJML, and Colt.
Ive had excellent luck with Colt for large scale calculations – but I’m not sure which ones implement the Kalaman method.