Question

How do I go about finding the running time (in Big-O notation) of the basic algorithm that performs (y − 1) multiplications by x to find x^y?

Edit: I also need to keep in mind the running time of each multiplication: “assuming that the time to multiply an n-bit number by an m-bit number is O(mn)“.

Answer 1

Well, you just have to consider the number of bits for each operation and sum them up. Of course we’ll do a little rounding in order to simply things since it won’t affect the big-O notation answer.

So, the number of bits in x is ceiling(log2(x)) (i.e. the next integer above the logarithm base 2 of x). We will call this number b, for simplicity. This means that x is bigger than 2^(b-1) and smaller than 2^b. So x^2 is bigger than 2^(2(b-1)) and smaller than 2^(2b). As such, we’re going to assume that x^2 is of size approximately 2b, and that in general, x^n is of size nb. This is pretty close to correct since it lies between n(b-1) and nb.

Our first multiplication will take time b*b = b^2. Our second multiplication will take 2b*b (since x^2 has size 2b and x is still size b). Our third will be 3b*b (since x^3 has size 3b and x is still size b). So, in general, our nth multiplication will be nb*b.

So our sum looks like b^2 + 2b^2 + 3b^2 + … + (y-1)b^2. We can factor out b^2, giving us (b^2)(1+2+3+…+(y-1)). For the second term, 1+2+3+…+(y-1) we can use a common formula that 1+2+3+…+n = n(n+1)/2. So we get (b^2)(y-1)(y)/2.

At this point we’re very close to the answer, but we want to do two things: we want to express everything in terms of x and y (and not b) and we want to reduce things using big-O notation for simplicity. (y-1)(y)/2 can be simplified to O(y^2). b = ceiling(log2(x)), which can be simplified to O(log(x)). Substituting back in, we get O( (log(x))^2 * y^2 ).

All this is, of course, assuming that we’re using an algorithm which looks like this:

product = 1
for i = 1 to y
  product = product * x
return product

If we’re doing something more complex and strange like this one:

xs = empty list
for i = 1 to y
  xs.addItem(x)
while xs is not empty
  num1 = xs.removeRandomItem
  num2 = xs.removeRandomItem
  xs.addItem(num1*num2)
return head(xs)

then the timing analysis gets way more complex. (Note that this algorithm also does y-1 multiplications and gets the correct result.)

The other common algorithm for finding x^y is the repeated squaring algorithm which works like this:

result = 1
temp = x
while y>0
  if y mod 2 = 1
    result = result * temp
    y = y - 1
  temp = temp * temp
  y = y / 2
return result

For that one we do two multiplications each round which involve two numbers, each of which are size b(2^n) where n is the round number counting from 0 (that is, the number of times we’ve been through the while loop). So in round n, each multiplication would cost b(2^n)*b(2^n)=(b^2)(2^(2n)). But it only goes ceiling(log2(y)) rounds. So its running time would be the sum of (b^2)(2^0)+(b^2)(2^2)+(b^2)(2^4)+…+(b^2)(2^(2*ceiling(log2(y)))). So again we can factor out b^2 leaving (b^2)(2^0+2^2+2^4+…+2^(2*ceiling(log2(y)))). Despite looking complicated, the whole second term is actually O(y^2). Once we again substitute for b, we find that this one is also O((log(x))^2 * y^2). So, although this algorithm is faster when multiplications are constant time operations, it’s not necessarily faster when we’re working with BigIntegers or other arbitrary precision types. This is why it’s most commonly used with situations like matrix multiplication where the cost of doing a multiplication is constant but large.