What is the fastest and most stable non-sql database to store big data and process thousands requests during the day (it’s for traffic exchange service)? I’ve found Kdb+ and Berkeley DB. Are they good? Are there other options?
More details…
Each day server processes > 100K visits. For each visit I need to read corresponding stats from DB, write log to DB and update stats in DB, aka 3 operations with DB per visit. Traffic is continuously increasing. Thus DB engine should be fast. From one side DB will be managed by demon written on C, Erlang or any other low-level language. From another side DB will be managed by PHP scripts.
Berkely DB is tried and tested and hardened and is at the heart of many mega-high transaction volume systems. One example is wireless carrier infrastructure that use huge LDAP stores (OpenWave, for example) to process more than 2 BILLION transactions per day. These systems also commonly have something like Oracle in the mix too for point in time recovery, but they use Berkeley DB as replicated caches.
Also, BDB is not limited to key value pairs in the simple sense of scalar values. You can store anything you want in the value, including arbitrary structures/records.