Home/ Questions/Q 7799855
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-02T00:26:26+00:00

I need a tool similar to cdb (constant database) that would allow me to

  • 0

I need a tool similar to cdb (constant database) that would allow me to store large sets of data (in the range of hundreds of gigabytes) in indexed files. CDB is an ideal candidate but it has a 2 GB file size limit so it’s not suitable.
The functionality I’m looking for is a persistent key-value store supporting binary keys and values. After creating the database is read only and will never be modified.
Can you recommend some tool? And btw, storage overhead should be small because I will be storing billion of records.

BTW I’m looking for a db management library (embeddable), not a standalone server. Something that can be used inside a C program.

Thanks,
RG

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Another option is mcdb, which is extended from Dan J. Bernstein’s cdb.

    https://github.com/gstrauss/mcdb/

    mcdb supports very large constant databases and is faster than cdb, both for database creation and database access. Still, creating a database of hundreds of gigabytes can take a bit of time. mcdb can create a gigabyte-sized database in a few seconds for cached data or in a minute or so when starting from cold cache.

    https://github.com/gstrauss/mcdb/blob/master/t/PERFORMANCE

    (Disclosure: I am the author of mcdb)

    • 0