Question

I have a bit counting method that I am trying to make as fast as possible. I want to try the algorithm below from Bit Twiddling Hacks, but I don’t know C. What is ‘type T’ and what is the python equivalent of (T)~(T)0/3?

A generalization of the best bit
counting method to integers of
bit-widths upto 128 (parameterized by
type T) is this:

v = v - ((v >> 1) & (T)~(T)0/3);      // temp 
v = (v & (T)~(T)0/15*3) + ((v >> 2) & (T)~(T)0/15*3);      // temp
v = (v + (v >> 4)) & (T)~(T)0/255*15;                      // temp
c = (T)(v * ((T)~(T)0/255)) >> (sizeof(v) - 1) * CHAR_BIT; // count

Answer 1

T is a integer type, which I’m assuming is unsigned. Since this is C, it’ll be fixed width, probably (but not necessarily) one of 8, 16, 32, 64 or 128. The fragment (T)~(T)0 that appears repeatedly in that code sample just gives the value 2**N-1, where N is the width of the type T. I suspect that the code may require that N be a multiple of 8
for correct operation.

Here’s a direct translation of the given code into Python, parameterized in terms of N, the width of T in bits.

def count_set_bits(v, N=128):
    mask = (1 << N) - 1

    v = v - ((v >> 1) & mask//3)
    v = (v & mask//15*3) + ((v >> 2) & mask//15*3)
    v = (v + (v >> 4)) & mask//255*15
    return (mask & v * (mask//255)) >> (N//8 - 1) * 8

Caveats:

(1) the above will only work for numbers up to 2**128. You might be able to generalize it for larger numbers, though.

(2) There are obvious inefficiencies: for example, ‘mask//15’ is computed twice. This doesn’t matter for C, of course, because the compiler will almost certainly do the division at compile time rather than run time, but Python’s peephole optimizer may not be so clever.

(3) The fastest C method may well not translate to the fastest Python method. For Python speed, you should probably be looking for an algorithm that minimizes the number of Python bitwise operations. As Alexander Gessler said: profile!