Question

I’m taking a course in computational complexity and have so far had an impression that it won’t be of much help to a developer.

I might be wrong but if you have gone down this path before, could you please provide an example of how the complexity theory helped you in your work? Tons of thanks.

Answer 1

O(1): Plain code without loops. Just flows through. Lookups in a lookup table are O(1), too.

O(log(n)): efficiently optimized algorithms. Example: binary tree algorithms and binary search. Usually doesn’t hurt. You’re lucky if you have such an algorithm at hand.

O(n): a single loop over data. Hurts for very large n.

O(n*log(n)): an algorithm that does some sort of divide and conquer strategy. Hurts for large n. Typical example: merge sort

O(n*n): a nested loop of some sort. Hurts even with small n. Common with naive matrix calculations. You want to avoid this sort of algorithm if you can.

O(n^x for x>2): a wicked construction with multiple nested loops. Hurts for very small n.

O(x^n, n! and worse): freaky (and often recursive) algorithms you don’t want to have in production code except in very controlled cases, for very small n and if there really is no better alternative. Computation time may explode with n=n+1.

Moving your algorithm down from a higher complexity class can make your algorithm fly. Think of Fourier transformation which has an O(n*n) algorithm that was unusable with 1960s hardware except in rare cases. Then Cooley and Tukey made some clever complexity reductions by re-using already calculated values. That led to the widespread introduction of FFT into signal processing. And in the end it’s also why Steve Jobs made a fortune with the iPod.

Simple example: Naive C programmers write this sort of loop:

for (int cnt=0; cnt < strlen(s) ; cnt++) {   /* some code */ } 

That’s an O(n*n) algorithm because of the implementation of strlen(). Nesting loops leads to multiplication of complexities inside the big-O. O(n) inside O(n) gives O(n*n). O(n^3) inside O(n) gives O(n^4). In the example, precalculating the string length will immediately turn the loop into O(n). Joel has also written about this.

Yet the complexity class is not everything. You have to keep an eye on the size of n. Reworking an O(n*log(n)) algorithm to O(n) won’t help if the number of (now linear) instructions grows massively due to the reworking. And if n is small anyway, optimizing won’t give much bang, too.