I have a two dimensional array of three dimensional volumes. This two dimensional array represents a top-down view of all of the three dimensional volumes.
I want to save this data to a file in such a way that I can retrieve it very quickly. My problem is that the two dimensional array may change size and shape; it’s not always nice and square. This tends to leave unused sections and quite a lot of them.
My method of retrieval is currently using the volume-level 2D view to locate volumes that need to be loaded, but I am having difficulty coming up with a good data structure and storage technique. Most of the methods I have come across require the 2D view to be of the same length and width or depend on the length or the width.
It may be worthy to note that I want to avoid unused space in the file and that I also want good locality for the mapped points. When mapping points, it is pretty common to come up with a solution which works but produces odd relationships; {0, 0} must not map to anything but {0} and {1, 0} should be pretty close to {0} and not something like {34}.
How would you go about doing this in a space and time efficient manner?
I solved this a while back by implementing a few different space filling curves and using them to map and transform the upper dimensional data to the single dimension file. I found that a Hilbert curve worked perfectly.