Analysis Framework (Sect 2.1)

• We will study two forms of algorithm efficiency
  • Time efficiency (or time complexity)
  • Space Efficiency (or space complexity)
• As time has gone by, the issues of time and space seem to have become much less critical
  • Processors have become MUCH faster
  • Memory has become MUCH cheaper, and more abundant.
• But the problems have become much much larger.
• So the efficiency of algorithms (and programs) is still an important area of study.
• We tend to focus on speed, as that is the easiest to improve
• But we will pay attention to space
  • As we have seen, this can go very wrong
• How much space do each of the GCD algorithms use?
• Input Size.
  • Most algorithms take more time/space as the size of the input increases
  • We use the input size as a parameter for a time/space function for algorithms.
    • The infamous "n".
    • But it could be two values (Graph G=(V,E), |V| and |E|
    • Or more.
  • We are looking for a measurement of the input which has an impact on the algorithm.
  • A function T(n) which predicts how long the algorithm will take to complete.
  • Input sizes for some problems
    • Searching an unsorted array
    • Evaluating a polynomial.
    • Finding the GCD of two numbers
    • Exploring all nodes in a graph.
    • Multiplying two matrices.
    • Flood Fill in a raster image.
• Units for Measuring Run Time
  • Wall clock time is just out
    • This includes the speed of a given computer.
    • And is not accurately measured.
    • Compiler optimizations can change algorithm performance.
    • Hardware optimizations can change algorithm performance.
    • Memory has an impact, as does ...
  • So we normally select a basic operation and count the number of times it is performed
    • Comparison in a search
    • Multiplication/Addition in polynomial evaluation
    • Division in GCD?
  • We could count all basic operations
    • But this would be architecture dependent
    • And it doesn't matter in the end anyway
  • He works through a quick example
    • Suppose on a given machine an operation takes c_op
    • Suppose that C(n) instructions are executed.
    • The time T(N) = c_opC(n)
    • Assume that C(n) = 1/2n(n-1)
    • How long would it take if we doubled the number of instructions?

      T(2n)    copC(2n)    1/2(2n)(2n-1)   2(2n-1)
      ----- =  -------- = ------------- = ------- ≈ 4
      T(n)     copC(n)      1/2(n)(n-1)     n-1
      	    

    • So the constants "wash out" in the end.
• Orders of Growth
  • We frequently encounter a set of functions in algorithm performance.
  • Look at table 2.1, page 46.
  • If T(N) = 2ⁿ+n ² + log₂(n), what is the dominant feature of that function?
  • By the way, log₂(n) I will sometimes write as lg(n)
  • An important relationship between different base logs.

    loga( n) = x or  ax = n
    so                logb(ax) = logb(n)
                      x logb(a) = logb(n)
    but x = loga(n)
    so                loga(n) logb(a) = logb(n)
    or                loga(n) = logb(n)/logb(a)
    let c=1/logb(a) then loga(n) = c logb(n).
           

  • So the difference between the log in two different bases is just a constant.
  • Anything larger than n^c for some constant c is called exponential
  • Problems with such a run time are intractable
  • Intractable problems with large input can not be solved in a reasonable amount of time.
• Worst Case, Best Case and Average Case Efficiencies
  • Consider the following algorithm

    SEQUENTIAL-SEARCH(A,K)
    
     i ← 0
     while i < n and A[i] ≠ K do
          i ← i -1
     if i < n return i
     else return -1
    
          

    • This algorithm takes an array A of n elements and a key K and determines if K is in A or not.
    • Note this uses C style array indexing (0 .. n-1)
    • Does it terminate?
    • Does it work?
    • How good is it?
    • What is the basic operation?
  • We consider the last question in three cases
    • Worst case, how bad can it be?
    • Best case, how good can it be?
    • Average case, where will most cases fall.
  • Worst: it will take n comparisons.
  • Best: it will take one comparison.
  • Average:
    • Assume the probability that K is in the array to be p
    • Assume that the array is random, ie the chance that K is at position i, 0 ≤ i < n, is p/n
    • T(n) = the cost if it is in the array + the cost if it is not.

      T(n) = [1*p/n + 2 * p/n + ... + n*p/n] + n*(1-p)
      T(n) = p/n(1 + 2 + 3 + ... + n) +n(1-p)
           = p/n*(n)(n+1)/2 + n(1-p)   (See Page 476)
           = 1/2 p(n+1) + n(1-p)
           ≈ n
      	   

  • Best and worst cases are "easy", Average is usually tough.