I’m currently developing a DynamoDB-backed application that stores customer-related data for a large customer base. It’s perfectly adapted for DynamoDB: customer IDs as hash keys partition well.
I have some related metadata that goes with this dataset. It’s quite small now (about 10 items) but I expect it to grow over the next few years to as many as 100. As small as it is, I’ve considered simply hardcoding it, but that will distance the data from the business stakeholders and needlessly complicate deployment (I’m developing a library to be used across several sites).
Therefore, because it makes sense to keep like things with like, it would make sense to keep the metadata in a DynamoDB table, alongside the data it metas. However, the DynamoDB docs warn against using a small number of popular hash keys, citing scalability concerns. Uncached, the small dataset is likely to use the same capacity estimate as the large dataset, though I would offset that with some aggressive caching.
What’s a conscientious developer to do?
If these metadata are intended to be used by heavy decison algorithms, it would probably make more sense to use memcached or some other distributed cache alternative, in addition, since the database would not be the bottleneck, you may choose any database you want.
The implementation would be very simple, just a proxy to the data access component, that would look into the cache for the key, when it is not found it reads from the database and next put into the cache.
Take a look at AWS ElasticCache