Question

I’m trying to wrap my head around the concept of studying an algorithm’s time complexity with respect to bit cost (instead of unit cost) and it seems to be impossible to find anything on the subject.

Here’s what I have so far:

The bit cost of multiplication and division of two numbers with n bits is, with arbitrary arithmetics, O(n^2).

So, for example:

int number = 2;
for(int i = 0; i < n; i++ ){
    number = i*i;
}

has a time complexity with respect to bit cost of O(log(n)^2*n), because it does n multiplications and i has log(i) bits.

But in a regular scenario we want the time complexity with respect to the input. So, how does that scenario work? The number of bits in i could be considered a constant. Which would make the time complexity the same as with unit cost except with a bigger constant (and therefore both would be linear).

Addition, subtraction, comparisons and assigning variables are all O(n), n being the number of bits, for arbitrary precision arithmetic.

Answer 1

With limited precision arithmetic (like the most-probably 32 bit int multiplication in your example) the bit cost of multiplication is constant. The cost of multiplying two int is O(32^2) using a naive multiplication algorithm (for better algos look here). This is the same as O(1) so people usually neglect to mention it when analysing algorithms.

If we use arbitrary precision arithmetic however, then it becomes important. If an arbitrarily long number with a value of i was stored in bits, it would take up O(log(i)) bits. So the cost of your code snippet would be O(log(n)^2 * n) (I’m using the fact that all i are no larger than n since your loop goes up to n).

As far as addition and subtraction goes I would say that they both have a bit cost of O(n), where n is the number of bits of the smaller operand.