Introduction to Algorithm Performance

Notes

• There are some of you who have been in class for a discussion of hardware performance.
  • Do we want to have anything change in hardware performance over time?
  • Or do we expect hardware to get faster?
• What about algorithms?
  • Does it make sense for INSERTION-SORT to get "faster" over the years?
  • Is this a measure of the algorithm or of the hardware running the algorithm?
• This is from Algorithm Design by Kleinberg and Tardos.
  • A good book I have used in the past.
• An algorithm is efficient if, when implemented, it runs quickly on real input instances.
  • On the surface this sounds great.
  • But the authors referenced, list a number of objections
    • Where? On what computer?
      • Do different instances of hardware have different performance characteristics?
    • Will this be stable over time?
      • Moore's law says we EXPECT computers speeds to increase over time.
    • Should the OS/Compiler/interpreter/language have an impact on algorithm performance?
      • Will these change over time?
      • Because a compiler is better, does that mean the algorithm improved?
    • How about implementation, could one programmer be better than another? How about compilers?
  • Just what is a "real input instance?"
    • What was huge 10 years ago is normal today ...
  • What does "runs quickly" mean?
• I am sure that if you think about it, you can find other objections.
• The authors state that we need a definition of performance that
  • Is platform independent
  • Instance independent (ie, we don't worry about implementation, hardware, language, compiler, ...)
  • Predicts performance as input scales up.
• What was that last one?
  • We have found sequential search to be O(n), and insertion sort to be O(n²)
  • Which is "faster" or if you have a choice, which would you rather do by hand?
    • In some sense this is a dumb question, as they do different things.
    • But if you walk in and I say, you can do one of these algorithms, which one, ...?
  • Look at desmos.com/calculator
    • f(x) = x
    • f(x) = x^2
  • If x is the size of the input (normally n in our case, but ...)
  • f(x) is the time it takes.
  • Do you want the first or second?
  • Change the first to be f(x) = 5x, does this matter?
• As the size increases, does one function "dominate" the other?
  • Is this reflective of "real life performance"?
• So far, determining the size of the input is easy
  • But we will see some problems where this is not true.
  • Be especially careful as we encounter graph algorithms.
  • Discuss TSP on a dense and sparse graph.
• You need to be sure you always state the input size, or "what is n?"
• What do we want to know?
  • Best Case?
  • Average Case?
  • Worst Case?
• Think about best case for both sequential search and insertion sort.
  • What are these?
  • What do they tell us about the algorithm?
• Average case is sometimes much harder to compute
  • And is probably not predictive of real input.
  • Real input is probably NOT completely random
• We tend to want to compute the worst case
  • It is frequently easy to compute (just like best case)
  • And provides information about how bad things can be
• A side trip into brute force or exhaustive searches
  • Remember This:

  • PERMUTATION-SORT(A)
    
     P = A
     while P is not sorted
        P = NEXT-PERMUTATION(P)
     return P

  • This is a brute force or exhaustive algorithm.
  • It will take O(n!) where n is the size of the array.
  • n! is a bad function, by the way.
  • n! will always reach a point where it is bigger than n^c, where c is a constant.
• In this case, the solution space is all of the permutations of the array.
• An algorithm is efficient if it achieves qualitatively better worst-case performance, at an analytical level, than a brute-force search of the solution space.
  • The idea is OK, but we still need to define analytical level and qualitatively better
• Eventually An algorithm is efficient if it has polynomial running time.