Analysis of Algorithms

• One approach
  • Code the solution in C++ (but we didn't like this before)
  • Translate this to assembly language
  • Count the number of times each instruction gets executed
  • Sum this up.
• But:
  • Yuck
  • This is lots of work.
  • And we are still looking at a specific implementation.
  • And what assembly language? (CISC/RISC/Stack Baised/...)
• So we define a set of primitive operations
  • Each counts as one operation.
  • It ignores hardware issues (memory access, cache, time for routine calls, ...)
  • This is a high level estimate, but is accurate
  • And we don't need to mess with assembly.
  • primitive operations
    • Assiging a value to a variable
    • Calling a function
    • Arithmetic operatioins like adding two numbers
    • Indexing an array, or following a pointer
    • Returning from a function.
  • Consider IsInOrder

     
         inorder = true                         (1) [1]
         i=0;                                   (1) [1]
    top: if(i>arraysize) goto out               (2) [size+1]
            if (array[i] > array[i+1] goto next (4) [size]
               inorder = false                  (1) [?]
    next:    i = i + 1                          (2) [size]
             goto top                           (1) [size]
    out:             
    
    We can assume either 
       i:  the array is in order and the 5th
           statement is always executed.
       ii: the array is not in order and the 5th
           statement is never executed.
       iii: Something inbetween.
    
       Execution time = 1*1+1*1 + 2*(size+1) + 4*size+ 0*1 + 2*size + 1*size
                      = 4+9*size
    	or
    	          = 4+10*size
           or 
                      = 4+9.5*size
           

• The above computation shows
  • The best case.
  • The worst case.
  • The average case.
• Computation may be diffacult depending on the algorithm and which case we are looking for.
• Best and worst cases sometimes depend on special input.
  • Consider sort3 in the example code.
  • What is it's best case?
  • What is it's worst case?

Asymptotic Notation

• In some sense does the 4 in the above computation matter?
  • Is it accurate on all computers?
  • What happens as size gets bigger (does it count as much?)
• How about the 9* or 10*?
• As the size of the array increases, this really dominates any computation of performance,
• and this is independant of the hardware/software.
• Big O notation.
  • Given f(x) and g(x), two functions that map nonnegative integers to real numbers.
  • We say that f(x) = O(g(x)) if for some value x₀ there exists a constant c>0 such that f(x)<= c*g(x) for all x>x₀
  • This essentually says, ignore all but the most sugnificant term, and ignore all constant multipliers.
  • we can say f(x) is Big-O of g(x)
  • we can say that f(n) is order g(n).
  • This really divides algorithms into rough sets of classes.

  • all polynomials of degree i are O(nⁱ)
  • If there is no input size, the algorithm is O(1)

  • This is not the entire truth, it looks at algorithms as the imput size increases.
  • The leading coefficient can mean that an O(n lg(n)) sort is worse in the short run than an O(n²), but when n gets "big enough", the first algorithm is always preferable.
  • Constant time algorithms - O(1)
  • Linear Algorithms - O(n)
  • Quadratic Algorithms - O(n²)
  • Polynomial Algorithms - O(n^k), where k is an integer
  • Exponential Algorithms - O(aⁿ), where a > 1
  • Big-O is bound from above.
• Omega is bound from below.
• Theta is bound above and below.
• There are also little-{o,theta, omega} definitions as well.