I am trying to use redis via c++ client in debian environment to store big lists of strings(nearly 2 million strings per list). I used to handle this using boost library’s unordered set, but since I have to persist each list to file system periodically and I get lots of updates and deletes instead of developing my db system, I wanted to gıve redis a go.
my redis configuration has no periodical saves whatsoever, I am planning to manually save all of the changes in the memory, when my update completes. But for test purposes, I am not even performing a save to disk. My test app performs 2 million writes, which takes close to 100 secs. This is pure memory operation since I am not performing any snapshot operations(I set configuration to not to save at all)
If I perform the same 2 million inserts using the unordered set it takes close to 5 seconds. Every tech site on the internet says, redis is super fast but in my case it is almost 50 times slower than boost’s unordered set even if no IO is involved, why is it so? Am I overlooking something obvious here?
You may want to read the Redis benchmark page
Redis is efficient for remote storage.
You are trying to compare local storage (boost::unordered_set) to remote storage (Redis). The cost of maintaining the set in memory is negligible compared to the cost of performing roundtrips to Redis. This is also true if your Redis instance is hosted on the same box. Any IPC is way more expensive than a hash table lookup.