Question

This may not be a programming question but it’s a problem that arised recently at work. Some background: big C development with special interest in performance.

I’ve a set of integers and want to test the membership of another given integer. I would love to implement an algorithm that can check it with a minimal set of algebraic functions, using only a integer to represent the whole space of integers contained in the first set.

I’ve tried a composite Cantor pairing function for instance, but with a 30 element set it seems too complicated, and focusing in performance it makes no sense. I played with some operations, like XORing and negating, but it gives me low estimations on membership. Then I tried with successions of additions and finally got lost.

Any ideas?

Answer 1

For sets of unsigned long of size 30, the following is one fairly obvious way to do it:

• store each set as a sorted array, 30 * sizeof(unsigned long) bytes per set.
• to look up an integer, do a few steps of a binary search, followed by a linear search (profile in order to figure out how many steps of binary search is best – my wild guess is 2 steps, but you might find out different, and of course if you test bsearch and it’s fast enough, you can just use it).

So the next question is why you want a big-maths solution, which will tell me what’s wrong with this solution other than “it is insufficiently pleasing”.

I suspect that any big-math solution will be slower than this. A single arithmetic operation on an N-digit number takes at least linear time in N. A single number to represent a set can’t be very much smaller than the elements of the set laid end to end with a separator in between. So even a linear search in the set is about as fast as a single arithmetic operation on a big number. With the possible exception of a Goedel representation, which could do it in one division once you’ve found the nth prime number, any clever mathematical representation of sets is going to take multiple arithmetic operations to establish membership.

Note also that there are two different reasons you might care about the performance of “look up an integer in a set”:

• You are looking up lots of different integers in a single set, in which case you might be able to go faster by constructing a custom lookup function for that data. Of course in C that means you need either (a) a simple virtual machine to execute that “function”, or (b) runtime code generation, or (c) to know the set at compile time. None of which is necessarily easy.
• You are looking up the same integer in lots of different sets (to get a sequence of all the sets it belongs to), in which case you might benefit from a combined representation of all the sets you care about, rather than considering each set separately.

I suppose that very occasionally, you might be looking up lots of different integers, each in a different set, and so neither of the reasons applies. If this is one of them, you can ignore that stuff.