Correctness of Insertion Sort

Notes

• INSERTION-SORT(A)
  
   for j = 2 to to A.length
      key = A[j]
      i = j-1
      while i > 0 and A[i] > key
          A[i+1] = A[i]
          i = i - 1
      A[i+1] = key
  

• Trace insertion sort for A = {5, 2, 4, 6, 1, 3};

  j  key  i             A
                1  2  3  4  5  6 
  2   2   1     5  2  4  6  1  3   Begin at line 4
          1     5  5  4  6  1  3   Line 5
          0     5  5  4  6  1  3   Line 6
          0     2  5  4  6  1  3   Line 7
  3   4   2     2  5  4  6  1  3   Line 4
          2     2  5  5  6  1  3   Line 5
          1     2  5  5  6  1  3   Line 6
          1     2  4  5  6  1  3   Line 7
  4   6   3     2  4  5  6  1  3   Line 4
          3     2  4  5  6  1  3   Line 7
  5   1   4     2  4  5  6  1  3   Line 4
          4     2  4  5  6  6  3   Line 5
          3     2  4  5  6  6  3   Line 6
          3     2  4  5  5  6  3   Line 5
          2     2  4  5  5  6  3   Line 6
          2     2  4  4  5  6  3   Line 5
          1     2  4  4  5  6  3   Line 6
          1     2  2  4  5  6  3   Line 5
          0     2  2  4  5  6  3   Line 5
          0     1  2  4  5  6  3   Line 7
  6   3   5     1  2  4  5  6  3   Line 4
          5     1  2  4  5  6  6   Line 5
          4     1  2  4  5  6  6   Line 6
          4     1  2  4  5  5  6   Line 5
          3     1  2  4  5  5  6   Line 6
          3     1  2  4  4  5  6   Line 5
          2     1  2  4  4  5  6   Line 6
          2     1  2  3  4  5  6   Line 7
  7             1  2  3  4  5  6 Termination 
      

• Notice that for each iteration of the loop
  • At the beginning of the loop (Lines 1-7) A[1 .. j-1], indicated in bold above, are in order.
    • Nothing is known about A[j .. A.length]
  • At the end of the loop A[1 .. j]) are in order and j increases by 1
    • Nothing is known about A[j+1 .. A.length]
  • Note that the outer loop (Lines 1-7) terminates when j = length
    • So at termination, A[1..j] are in order.
• So what does this algorithm do?
  • It continues to grow a sorted array by "inserting" the next element from the unsorted array into the sorted portion. It does this by moving the elements in the sorted array up one position thus making a free position where the new element must go.
• They formalize this to be a loop invariant or a property that is true after every iteration of the loop
  • At the star of each iteration of the for loop of lines 1-7, the subarray A[i..j-1] consists of elements originally in A[1..j-1] but in sorted order.
• We can prove that the loop invariant holds at the beginning of execution by observing
  • When j = 2, A[1..j-1] is A[1 ..1], which contains a single element, which is in order.
• Next we consider Maintenance, or at the end of each execution of the loop, does the loop invariant hold.
  • Formally we should come up with a loop invariant for loop 4-6
  • Each element A[i+1 .. j] is in order and larger than the key, and A[i+1] is empty
  • And we should repeat the process we are performing for the outer loop.
  • Informally we can state that each element in the ordered array is moved up one position until we reach the end of the array or the element is not larger than the key. (lines 4-6)
  • Then when the loop terminates, the key is returned to the correct position (line 7)
  • In either case, we can claim the loop invariant is maintained.
• Finally we need to argue termination
  • When j = A.length+1, we
    • exit the for loop in line 1
    • But we know from the loop invariant that A[1 .. A.length] is in order from the loop invariant.
    • The the entire array is in order.