I am looking for a reasonably well tested library+server to store a persistent distributed hash table.
I am hesistant to use SQL-based solutions as the data is highly document oriented, consisting of millions of ~64KB blobs with only a single index (computed by hash of said BLOB) – and needs to be able to be distributed for long term scaling prospects.
Due to expense and bandwidth considerations, external solutions such as S3 are not an option.
Something like CouchDB or Project Voldemort would be ideal – however there is a noticable lack of .NET bindings for both (PV can be IKVMC’d from Java – however has “issues”.). Both key and value are byte arrays (key is 16 byte, the value is up to 2048KB averaging 64KB)
I have searched so far for some kind of .NET port of Dynamo, Chord and similar – however the majority of results appear to be purely in-memory caches and lack any form of persistence or replication.
Anyone got any ideas or suggestions?
Take a look at Ayende’s Rhino DHT. Might be more inline with what you are looking for. The source can be acquired here.