Question

I’m developing a Java application that has performance at its core.
I have a list of some 40,000 “final” objects,
i.e., I have an initialization input data of 40,000 vectors.
This data is unchanged throughout the program’s run.

I am always preforming lookups against a single ID property to retrieve the proper vectors.
Currently I am using a HashMap over a sub-sample of a 1,000 vectors,
but
I’m not sure it will scale to production.

When is BIG, actually big enough for a use of DB?
One more thing, an SQLite DB is a viable option as no concurrency is involved,
so I guess the “threshold” for db use, is perhaps lower.

Answer 1

I think you’re asking whether a HashMap with 40,000 entries in will be okay. The answer is yes – unless you really don’t have enough memory, that should be absolutely fine. If you’re writing a performance-sensitive app, then putting a large amount of fast memory in the machine running the app is likely to be an efficient way of boosting performance anyway.

There won’t be very much overhead for each HashMap entry, so if you’ve got enough space to store the objects themselves in memory, it’s unlikely that the overhead of the map would cause a problem.

Is there any reason why you can’t just test this with a reasonable amount of data?

If you really have no more requirements than:

• Read data at start-up
• Put data in a map by a single ID (no need for joins, queries against different fields, substring matches etc)
• Fetch data from map

… then using a full-blown database would be a huge amount of overkill, IMO.