Performance of Insertion Sort

Notes

• INSERTION-SORT(A)
  
   for j = 2 to to A.length
      key = A[j]
      i = j-1
      while i > 0 and A[i] > key
          A[i+1] = A[i]
          i = i - 1
      A[i+1] = key
  

• For now, we will stick with what you may have learned about algorithm analysis.
  • How long will this algorithm take?
  • And I bet you did this by counting the instructions.
• So let's give this a try.
• Let's assume that there are n elements in the array.
• Line 1
  • Probably is close to

  • A.  i = 2
    B.  top: 
    C.       if (i > A.Length) jump to end
    D.           body of loop (Lines 2-7 above)
    E.           ++i
    F.           jumpto top
    G.  end:
    

  • 1A will execute one time
  • 1C will execute for i=2, 3, 4, 5, ... n, n+1
  • 1D-1F will execute for i=2, 3, 4, ... n
• So the body will execute n-1 times.
• But what about the body?
  • If the item at position i is the largest so far,
    • Line 4 will execute exactly once
  • If item the smallest so far
    • Line 4 will execute i+1 times
    • And lines 5,6 will execute i times.
• How do we deal with this?
  • In general we are interested in best, worst, and sometimes average time
  • In the best case, the array is in order, and we will execute the body (2-7) one time
  • This is O(n)
  • In the worst case, the array is completely reversed, and we will execute 4-6 1, 2, 3, 4, ... n times
  • This is harder.
• Our friend Gauss
  • (But not really, look at Triangular number on wikipedia)
  • $\sum_{i=1}^n i = \frac{n(n+1)}{2} = \frac{1}{2}n^2 + \frac{1}{2}n$
  • Which is $O(n^2)$
• Oh ya? Prove It!
  • I will use induction:
    • Prove the base case (n=0)
    • Assume true for n = k
    • Prove true for n = k+1
  • Statement :
    • $\sum_{i=1}^n i = \frac{n(n+1)}{2}$
  • Prove the base case, n = 1:
    • For n = 1
    • $\sum_{i=1}^1 = 1 $ $$ \begin{aligned} \frac{n(n+1)}{2} = \frac{1(1+1)}{2} \\ = \frac {1(1+1)}{2} \\ = \frac{1\times2}{2} \\ = 1 \end{aligned}$$
  • Assume true for n =k ≥ 1
    • $\sum_{i=1}^k i = \frac{k(k+1)}{2}$
  • Prove true for n = k+1
    • $\sum_{i=1}^{k+1} i = \frac{(k+1)(k+2)}{2}$
    • $$ \begin{aligned} \sum_{i=1}^{k+1} i = \sum_{i=1}^k i + (k+1) \\ = \frac{k(k+1)}{2} + k+1 \\ = \frac{k(k+1)}{2} + \frac{2(k+1)}{2} \\ = \frac{k^2+k +2k+2}{2} \\ = \frac{k^2+3k+2}{2} \\ = \frac{(k+1)(k+2)}{2} \\ \end{aligned}$$