Chapter 2, Algorithm Analysis

• Asymptotic bounds:
• f(n) is O(g(n)) if for constant c> 0, there is a n₀, such that f(n)≤ c g(n) when n >n₀
• We sometimes write that f(n)=O(g(n))
• This is bound from above
• f(n) is &Omega(g(n)) if for constant c> 0, there is a n₀, such that f(n)≥ c g(n) when n >n₀
• This is bound from below
• f(n) is Θ(g(n)) if f(n) = O(g(n)) and f(n) = Ω(g(n))
• Notice that only the Θ bound must be tight.
  • Linear search, as we have seen is O(n)
  • Binary search is also O(n), but is it O(lg(n)) as well.
  • Binary search is Θ(lg(n)) in the worst case.
  • We usually like as tight a bound as possible.
• The function n² + 1000n
  • is dominated by the 1000n term until n > 1000
  • After this n² dominates.
  • So n₀ is 1000
  • And c is 2.
  • and thus this is O(n²)
• Some observations:
  • If f₁(n) = O(g(n)) and f₂(n) = O(h(n))
    • f₁(n)+f₂(n) = max(O(g(n)),O(h(n)))
    • IE if we have one algorithm followed by another
    • f₁(n)*f₂(n) = O(g(n)*h(n)))
    • IE if we have one algorithm inside of another.
  • Figure 2.1, page 45
    • O(c) is constant time
    • O(lg n) is logarythmic time
    • O(lg² n) is log-squared time
    • O(n) is linear time
    • O(n lg n)
    • O(n²) is quadratic
    • O(n³) is cubic
    • O(n!) is factorial time.
    • O(aⁿ), a>1) is exponential
    • The last two are generaly classes considered to be intractable
  • As we should have seen in our first homework, finding the k^th item is O(n²) in the worst case.
    • No matter where we run it, it has the same curve
  • We can perform the analysis on
    • An algorithm, as with linear search
    • We will even look at the best canse performance for a problem.
    • On time performance, as in all of the above
    • On space performance, again consider fibinaci algorithms
    • Best, worst and average cases.
    • Remember, it does not always depend on input size:
      • The k^th item could depend on the input size as well as which item we were looking for.
      • Fibinaci was just which number in the sequence we were searching for.
  • Remember, sometimes we may have hidden costs:
    • An array copy, A = B , is O(n), where in is the size of the array.
    • This can occur if an array is passed by value.
• An additional Example:
  • Consider evaluating a polynomial in the form c_nxⁿ+c_n-1x^n-1 + ... + c₁x+c₀
  • What would your performance expectations be?
  • How would you store this polynomial? (an array of doubles)
    Consider algorithm 1

    Evaluate(coeff, n, x)
    
     value <- 0;
     for i<- 0 to n do
         value <- value + coeff[i] * xi
     return value
    
    	 

    • Give best, worst and average case for this algorithm.
    • Consider the following

      Horner(coeff, n, x) 
      
       value <- coeff[n]
       for i <- n-1 to 0 do 
          value <- value * x + coeff[i]
       return value
      
      	 

    • All the normal stuff, does it work, is it right, how does it work, what is the performance?