Question

I am watching Intro to Algorithms (MIT) lecture 1. Theres something like below (analysis of merge sort)

T(n) = 2T(n/2) + O(n)

Few questions:

• Why work at bottom level becomes O(n)? It said that the boundary case may have a different constant … but I still don’t get it …
• Its said total = cn(lg n) + O(n). Where does O(n) part come from? The original O(n)?

Answer 1

Although this one’s been answered a lot, here’s one way to reason it:

If you expand the recursion you get:

t(n) = 2 * t(n/2) + O(n)
t(n) = 2 * (2 * t(n/4) + O(n/2)) + O(n)
t(n) = 2 * (2 * (2 * t(n/8) + O(n/4)) + O(n/2)) + O(n)
  ...
t(n) = 2^k * t(n / 2^k) + O(n) + 2*O(n/2) + ... + 2^k * O(n/2^k)

The above stops when 2^k = n. So, that means n = log_2(k).

That makes n / 2^k = 1 which makes the first part of the equality simple to express, if we consider t(1) = c (constant).

t(n) = n * c + O(n) + 2*O(n/2) + ... + (2^k * O(n / 2^k))

If we consider the sum of O(n) + .. + 2^k * O(n / 2^k) we can observe that there are exactly k terms, and that each term is actually equivalent to n. So we can rewrite it like so:

t(n) = n * c + {n + n + n + .. + n}    <-- where n appears k times
t(n) = n * c + n *k

but since k = log_2(n), we have

t(n) = n * c + n * log_2(n)

And since in Big-Oh notation n * log_2(n) is equivalent to n * log n, and it grows faster than n * c, it follows that the Big-O of the closed form is:

O(n * log n)

I hope this helps!

EDIT

To clarify, your first question, regarding why work at the bottom becomes O(n) is basically because you have n unit operations that take place (you have n leaf nodes in the expansion tree, and each takes a constant c time to complete). In the closed-formula, the work-at-the-bottom is expressed as the first term in the sum: 2 ^ k * t(1). As I said above, you have k levels in the tree, and the unit operation t(1) takes constant time.

To answer the second question, the O(n) does not actually come from the original O(n); it represents the work at the bottom (see answer to first question above).

The original O(n) is the time complexity required to merge the two sub-solutions t(n/2). Since the time complexity of the merge operation is assumed to grow (or decrease) linearly with the size of the problem, that means that at each level you will have a sum of O(n / 2^level), of 2^level terms; this is equivalent to one O(n) operation performed once. Now, since you have k levels, the merge complexity for the initial problem is {O(n) at each level} * {number of levels} which is essentially O(n) * k. Since k = log(n) levels, it follows that the time complexity of the merge operation is: O(n * log n).

Finally, when you examine all the operations performed, you see that the work at the bottom is less than the actual work performed to merge the solutions. Mathematically speaking, the work performed for each of the n items, grows asymptotically slower than the work performed to merge the sub-solutions; put differently, for large values of n, the merge operation dominates. So in Big-Oh analysis, the formula becomes: O(n * log(n)).