A lower limit on comparison based sorting.

Objectives

1. Establish a lower bound on comparison based sorting.

Notes

• This is from clrs.
• A decision tree is the binary tree that represents the "decisions" that must be made to reach a conclusion.
  • In a decision tree, the leaves are "conclusions"
  • The inner nodes are the decisions required to reach those conclusions.
  • The tree is binary, because the questions are all yes-no.
• It is reasonable to model comparison based sorting with a decision tree.
  • Inner nodes are the comparisons
  • The leaves are a permutation of the data that is in order.
• To simplify the math a bit, we will assume that no two data elements are the same
  • This is to make each permutation unique.
• We know that for a set of n elements, there are n! permutations.
• Or, in other words, there are n! different ways to "sort" a set of unique elements.
  • But only one is "correct".
• So for a comparison based sort of n elements , a decision tree must support n! leaves.
• Therefore the height of the decision tree must be at least height h such that n! ≤ 2^h.
• or log₂n! ≤ h
• But we want to know this in terms of n, not h.
• Sterling's approximation gives us the relationship
  • n! ≈ sqrt(2πn) nⁿ/eⁿ
  • n! ≈ (2πn)^1/2 nⁿe^-n
  • log₂n! ≈ log₂(2π n)^1/2 nⁿe^-n)
    • log(ab) = log(a) + log(b)
  • log₂n! ≈ log₂(2πn)^1/2 + log₂ nⁿ - loge^-n
    • sqrt(ab) = sqrt(a)sqrt(b), so sqrt(2πn) = sqrt(2π) sqrt(n)
    • let sqrt(2π) = c
    • sqrt(n) = n^1/2
    • log₂n^1/2 = 1/2 log₂n
  • So log₂(n!) ≈ c 1/2 log₂n + n log₂n - n log₂ e
  • log₂n! ≈ n log₂n - n log₂e ++ O(log₂n)
  • Or log₂n! ≈ O(n log₂ n )
• So O(n log₂n) ≤ h, the height of the tree.
• Thus there must be at least O(n log₂ n) comparisons in any decision tree that finds the correct permutation.
• And the best ANY comparison based sort can be is O(n log₂n)