Question

I am learning algorithm analysis. I am having trouble understanding the difference between O, Ω, and Θ.

The way they’re defined is as follows:

• f(n) = O(g(n)) means c · g(n) is an
  upper bound on f(n). Thus there exists
  some constant c such that f(n) is
  always ≤ c · g(n), for large enough n
  (i.e., n ≥ n0 for some constant n0).
• f(n) = Ω(g(n)) means c · g(n) is a
  lower bound on f(n). Thus there exists
  some constant c such that f(n) is
  always ≥ c · g(n), for all n ≥ n0.
• f(n) = Θ(g(n)) means c1 · g(n) is an upper bound on f(n) and c2 · g(n) is a
  lower bound on f(n), for all n ≥ n0.
  Thus there exist constants c1 and c2
  such that f(n) ≤ c1 ·g(n) and f(n) ≥
c2 ·g(n). This means that g(n)
  provides a nice, tight bound on f(n).

The way I have understood this is:

• O(f(n)) gives worst case complexity of given function/algorithm.
• Ω(f(n)) gives best case complexity of given function/algorithm.
• Θ(f(n)) gives average case complexity of given function/algorithm.

Please correct me if I am wrong. If it is the case, time complexity of each algorithm must be expressed in all three notations. But I observed that it’s expressed as either O, Ω, or Θ; why not all three?

Answer 1

It is important to remember that the notation, whether O, Ω or Θ, expresses the asymptotic growth of a function; it does not have anything intrinsically to do with algorithms per se. The function in question may be the “complexity” (running time) of an algorithm, either worst-case, best-case or average-case, but the notation is independent of where the function comes from.

For example, the function f(n)=3n²+5 is:

• O(n²), it is also O(n²log n), O(n³), O(n⁴) and so on, but is not O(n).
• Ω(n²), it is also Ω(n log n), Ω(n) and so on, but is not Ω(n³).
• Θ(n²). It is not even Θ(n²log n) or Θ(n²/log n).

Now, usually the function considered is the worst-case complexity of an algorithm, and which notation of the three is used depends on what we want to say about it and on how carefully we do the analysis. For example, we may observe that because there are two nested loops, the worst-case running time is at most O(n²), without caring about whether this is actually achieved for some input. (Usually it is obvious that it is.) Or, we may say that the worst-case running time of sorting is Ω(n log n), because there must be some inputs for which it must take at least cn(log n) steps. Or, we may look at a particular mergesort algorithm, and see that it takes at most O(n log n) steps in the worst-case and that some input makes it take n log n steps, so the worst-case running time is Θ(n log n).

Note that in all the three examples above, it was still the same (worst-case) running time that was being analyzed. We may analyze the best-case or average-case instead, but again, which notation of the three we use depends on what we want to say — whether we want to give an upper bound, lower bound, or tight bound on the order of growth of the same function.