I have an open source library which is designed to support very large collections efficiently. I have tested it on my PC with 24 GB but would like to test it on much a larger system, up to 1 TB. Most hosting solutions don’t support this sort of memory sizes and I only need access for brief periods of time.
What I have tested is a collection of 500 million objects with 12 fields the full GC time is below 0.11 seconds. I have another test were it stores 128 billion elements with just one bit.
The library/test is small, so I don’t have require much other than a lot of main memory.
Do you have any suggestions on how I could do this testing without buying my own server with 96 or 192 GB?
EC2 has high-memory instances of up to 68.4 GB each, and they charge by the hour. Granted, that is not 100GB of memory, but, if you stack a few of them up together….