Question

What would be an efficient implementation for a std::set insert member function? Because the data structure sorts elements based on std::less (operator < needs to be defined for the element type), it is conceptually easy to detect a duplicate.

How does it actually work internally? Does it make use of the red-back tree data structure (a mentioned implementation detail in the book of Josuttis)?

Implementations of the standard data structures may vary…

I have a problem where I am forced to have a (generally speaking) sets of integers which should be unique. The length of the sets varies so I am in need of dynamical data structure (based on my narrow knowledge, this narrows things down to list, set). The elements do not necessarily need to be sorted, but there may be no duplicates. Since the candidate sets always have a lot of duplicates (sets are small, up to 64 elements), will trying to insert duplicates into std::set with the insert member function cause a lot of overhead compared to std::list and another algorithm that may not resort to having the elements sorted?

Additional: the output set has a fixed size of 27 elements. Sorry, I forgot this… this works for a special case of the problem. For other cases, the length is arbitrary (lower than the input set).

Answer 1

When you only have 64 possible values known ahead of time, just take a bit field and flip on the bits for the elements actually seen. That works in n+O(1) steps, and you can’t get less than that.

Inserting into a std::set of size m takes O(log(m)) time and comparisons, meaning that using an std::set for this purpose will cost O(n*log(n)) and I wouldn’t be surprised if the constant were larger than for simply sorting the input (which requires additional space) and then discarding duplicates.

Doing the same thing with an std::list would take O(n^2) average time, because finding the insertion place in a list needs O(n).

Inserting one element at a time into an std::vector would also take O(n^2) average time – finding the insertion place is doable in O(log(m)), but elements need to me moved to make room. If the number of elements in the final result is much smaller than the input, that drops down to O(n*log(n)), with close to no space overhead.

If you have a C++11 compiler or use boost, you could also use a hash table. I’m not sure about the insertion characteristics, but if the number of elements in the result is small compared to the input size, you’d only need O(n) time – and unlike the bit field, you don’t need to know the potential elements or the size of the result a priori (although knowing the size helps, since you can avoid rehashing).