Introduction to Algorithm Analysis

• On page 274 they introduce Analysis of Algorithms.
• This is continued on 693.
  • This is done formally in Algorithm Analysis (Current 385, future?)
• We need to know how to determine which algorithm might be "better"
  • The only real way to do this is to run the algorithm under operating conditions.
  • But we have come up with an approximate alternative
• Counting instructions
  • A reasonable approximation for time is to count the number of instructions executed.
  • But this depends on quite a few things.
    • The efficiency of the implementation
    • The instruction set of the machine
  • Also, we don't really code in assembly that much any more.
  • So we sort of punt and fake it.
• Counting loops (eeek, this is wrong but will do for now)
  • The fallback is to count the number of times the algorithm loops
  • But not all loops, only those related to the input.
  • This is easy for our algorithms
  • But becomes much more complex as we go along.
• What to count
  • As above, we count the number of operations on the input, or data that could change.
  • This is easy in our case, it is the size of the array.
  • As the array gets bigger, the number of iterations increase.
  • We will call the size of the data n in our discussion
• Big-O notation
  • In the end, we usually come up with some form of a polynomial to represent the number of instructions executed
  • $t(n) = \frac{3}{4}n^2 + 4n + 34$
  • We really don't care about most of this, so
    • We throw away everything but the highest order term.
    • And we discard the coefficient.
  • So the above $t(n) \approx n^2$
• Least, most, average
  • Some algorithms will perform different.
  • Think about sequential search
    • What is the best case?
    • What is the worst case?
    • What is the average case?
• If we say an algorithm is $O(f(n))$ we say it is no worse than $f(n)$
• Linear Search

  • LinearSearch(A, key)
    
     current ← 0
     while current < size and A[current] ≠ key
        current ← current + 1
     return current
     

  • In this case, n is the number of elements in the array.
  • How many times will the loop execute in worst case?
  • How about best case?
  • How about on average?
  • This algorithm is $O(n)$
• Selection Sort

  • Sort(A[], size)
    
     for current ← 0 to size-2
        smallest ← current
        for search ← current+1 to size-1
           if A[smallest] > A[search]
               smallest ← search
        if current ≠ smallest
           swap(A[current], A[smallest])
     

  • This is a little harder to see
  • But
    • line 1 will execute n-1 times.
    • Line 3 will execute $n-1 + n-2 + n-3 + ... + 3 + 2 + 1$ times
    • So the entire thing will execute $ \sum_{i=1}^{n-1}i$

    • $\sum_{i=1}^{n-1}i$	$ = \frac{(n-1)(n)}{2}$
      	$ = \frac{n^2 - n +1 }{2} $
      	$ = \frac{1}{2}n^2 - \frac{1}{2}n + \frac{1}{2} $
      	$ \approx n^2 $

    • So this is $O(n^2)$
• Binary Search

  • BinarySearch(A, low, high, key)
    
     found ← false
     while low ≤ high and not found
        mid ← ( low + high) / 2    
        if A[mid] = key
             found ← true
        else if A[mid] < key
             low ← mid+1
        else
             high ← mid-1
     if not found
        mid ← high+1
     return mid 
     

  • This one is much tougher.
  • One way to think about this is that the data is cut in half each time.
  • Assume we have 1024 data items.

  • Step	Items to Search
    1	1024
    2	512
    3	256
    4	128
    5	64
    6	32
    7	16
    8	8
    9	4
    10	2
    11	1

  • This is not quite right, but $O$ notation is sloppy anyway
  • We can deduce that this algorithm is $O(\log_2 n )$
  • Or $O(\log n)$
• See timeMaker.cpp