So I’m designing a matrix for a computer vision project and believe I have one of my calculations wrong. Unfortunately, I’m not sure where it’s wrong.
I was considering creating a matrix that was 100,000,000 x 100,000,000 with each ‘cell’ containing a single integer (1 or 0). If my calculations are correct, it would take 9.53674316 × 10^9 MB. Is that correct?!?
My next question is, if it IS correct, are there ways to reduce memory requirements to a more realistic level while still keeping the matrix the same size? Of course, there is a real possibility I won’t actually need a matrix that size but this is absolute worse case scenario (as put forth by a friend). The size seems ridiculous to me since we’d be covering such a small distance at a time.
Thanks1
Anthony
In theory, an element of {0, 1} should consume at most 1 bit per cell. That means 8 cells per byte or 1192092895 megabytes or about one petabyte, which is too much, unless you are google 🙂 Not to mention, even processing (or saving) such matrix would take too much time (about a year I’d say).
You said that in many cases you won’t even need matrix so large. So, you can create smaller matrix at start (10,000 x 10,000) and then double the size every time enlargment is needed, copying old contents.
If your matrix is sparse (has much much more 1’s than 0’s or vice-versa), then it is much more efficient to store just coordinates where ones are in some efficient data structure, depending what operations (search, data access) you need.
Side note: In many languages, you have to take proper care for that to be true, for example in C, even if you specify variable as boolean, it still takes one byte, 8 times as much as needed.