Question

I’m reading Introduction to Algorithms book, second edition, the chapter about Medians and Order statistics. And I have a few questions about randomized and non-randomized selection algorithms.

The problem:
Given an unordered array of integers, find i’th smallest element in the array

a. The Randomized_Select algorithm is simple. But I cannot understand the math that explains it’s work time. Is it possible to explain that without doing deep math, in more intuitive way? As for me, I’d think that it should work for O(nlog n), and in worst case it should be O(n^2), just like quick sort. In avg randomizedPartition returns near middle of the array, and array is divided into two each call, and the next recursion call process only half of the array. The RandomizedPartition costs (p-r+1)<=n, so we have O(n*log n). In the worst case it would choose every time the max element in the array, and divide the array into two parts – (n-1) and (0) each step. That’s O(n^2)

The next one (Select algorithm) is more incomprehensible then previous:

b. What it’s difference comparing to previous. Is it faster in avg?

c. The algorithm consists of five steps. In first one we divide the array into n/5 parts each one with 5 elements (beside the last one). Then each part is sorted using insertion sort, and we select 3rd element (median) of each. Because we have sorted these elements, we can be sure that previous two <= this pivot element, and the last two are >= then it. Then we need to select avg element among medians. In the book stated that we recursively call Select algorithm for these medians. How we can do that? In select algorithm we are using insertion sort, and if we are swapping two medians, we need to swap all four (or even more if it is more deeper step) elements that are “children” for each median. Or do we create new array that contain only previously selected medians, and are searching medians among them? If yes, how can we fill them in original array, as we changed their order previously.

The other steps are pretty simple and look like in the randomized_partition algorithm.

Answer 1

The randomized select run in O(n). look at this analysis.

Algorithm :
Randomly choose an element
split the set in "lower than" set L and "bigger than" set B
if the size of "lower than" is j-1 we found it
if the size is bigger, then Lookup in L
or lookup in B

The total cost is the sum of :

• The cost of splitting the array of size n
• The cost of lookup in L or the cost of looking up in B

Edited: I Tried to restructure my post

You can notice that :

• We always go next in the set with greater amount of elements
• The amount of elements in this set is n - rank(xj)
• 1 <= rank(xi) <= n So 1 <= n - rank(xj) <= n
• The randomness of the element xj directly affect the randomness of the number of element which
  are greater xj(and which are smaller than xj)

if xj is the element chosen , then you know that the cost is O(n) + cost(n - rank(xj)). Let’s call rank(xj) = rj.

To give a good estimate we need to take the expected value of the total cost, which is

T(n) = E(cost) = sum {each possible xj}p(xj)(O(n) + T(n - rank(xj)))

xj is random. After this it is pure math.
We obtain :

T(n) = 1/n *( O(n) + sum {all possible values of rj when we continue}(O(n) + T(n - rj))) )
T(n) = 1/n *( O(n) + sum {1 < rj < n, rj != i}(O(n) + T(n - rj))) )

Here you can change variable, vj = n – rj

T(n) = 1/n *( O(n) + sum { 0 <= vj <= n - 1, vj!= n-i}(O(n) + T(vj) ))

We put O(n) outside the sum , gain a factor

T(n) = 1/n *( O(n) + O(n^2) + sum {1 <= vj <= n -1, vj!= n-i}( T(vj) ))

We put O(n) and O(n^2) outside, loose a factor

T(n) = O(1) + O(n) + 1/n *( sum { 0 <= vj <= n -1, vj!= n-i} T(vj) )

Check the link on how this is computed.

For the non-randomized version :

You say yourself:
In avg randomizedPartition returns near middle of the array.

That is exactly why the randomized algorithm works and that is exactly what it is used to construct the deterministic algorithm. Ideally you want to pick the pivot deterministically such that it produces a good split, but the best value for a good split is already the solution! So at each step they want a value which is good enough, “at least 3/10 of the array below the pivot and at least 3/10 of the array above”. To achieve this they split the original array in 5 at each step, and again it is a mathematical choice.