Chapter Introduction

• Please read chapter 2.
• When we discuss efficiency we frequently think of time only
  • But other resources can be an issue.
  • Especially space.
• How can we define efficiency?
  • For the moment forget the big O thing.
• They go through a number of potential definitions for efficiency
  • An algorithm is efficient if, when implemented, runs quickly on real input instances.
    • Implementation is ALWAYS an issue
      • Where will you implement the algorithm?
        • This is very likely to change over the years.
        • Is it reasonable to have a measurement that is bound by a given time period.
        • How many DEC VAX-11/780 computers do you see these days?
        • From Wikipedia:

          (In the VAX 780 article) Digital used the performance of the VAX-11/780 as a reference point for describing the performance of subsequent VAX models. The performance of the VAX-11/780 became known as 1 VAX Unit of Performance (or 1.0 VUPs).[8] Other VAX models were rated as a multiple of the VAX-11/780's performance, for example, a 2.0 VUPs VAX would be twice as fast as the VAX-11/780.

          (In the VAX article) For a while the VAX-11/780 was used as a standard in CPU benchmarks. It was initially described as a one-MIPS machine, because its performance was equivalent to an IBM System/360 that ran at one MIPS, and the System/360 implementations had previously been de facto performance standards. The actual number of instructions executed in 1 second was about 500,000, which led to complaints of marketing exaggeration. The result was the definition of a "VAX MIPS," the speed of a VAX-11/780; a computer performing at 27 VAX MIPS would run the same program roughly 27 times faster than the VAX-11/780.

          [8] "VAX CPU Model Summary". vaxmacro.de. Retrieved 2020-12-27.

      • The quality of your implementation
        • How do you know you have the "best" implementation of an algorithm?
        • By the way, this changes for different hardware as well.
          • With different instruction sets.
          • Different memory access models
          • Different cache
          • Different levels of compiler optimization
          • And even different compilers.
      • Input
        • What does real input mean?
        • How do we know what the right size is?
          • In my experience, this tends to be limited by processor power, not the other way around.
        • Plus Do we know all of the cases.
      • They make a point that we want something that is
        • Platform independent
        • Instance independent
        • Predicts as the input grows.
  • A side note: Worst Case and Brute-Force
    • Note we have frequently been considering the worst case scenario.
    • This turns out to be reasonable, but not the only measurement.
      • Average case can become a problem.
      • Best case is often over optimistic
    • Brute force is usually the most obvious approach to many problems.
      • This is usually a "try everything and see what works" algorithm
      • And is is usually unsatisfactory.
      • Or alternatively generate every possible solution (the solution space)
      • And search this for the best possible solution.
    • Think about Stable Matching
      • There are n men and n women, so there are n! perfect matches.
      • Convince me of this!
    • So it would take n! operations to generate the entire search space.
      • In case you are not sure of it, n! gets big quickly.
      • try this
      • Or this in bc

        define f(n) {
          if (n < 2) return(1)
          return(n * f(n - 1))
        } 

    • He points out that we decided stable matching could be much faster than n! by analysis, not experimentation.
    • And usually brute-force is bad for anything but the simplest computations.
  • An algorithm is efficient if it achieves qualitatively better worst-case performance, at an analytical level, than a brute-force algorithm.
    • The problem with this is qualitatively better
    • Search spaces tend to be either factorial or exponential based on an input size.
  • An algorithm is efficient if it has a polynomial running time
    • To start with, let's introduce a primitive computation step
      • Intuitively this could be a single assembly instruction.
      • Or a single line of C++ or java code. (erm, be careful here)
    • We want to count the number of these primitive computations with respect to the input.
    • More specifically, we would like a function like $T(n) = cn^d, c>0, d> 0$ to describe the computation.
      • For an input size of $n, T(n) = cn^d$
      • For an input size of $2n, T(2n) = c(2n)^d = cn^d2^d = 2^dT(n)$
      • Thus the length of time to solve the problem changes by a constant $2^d$.
      • Note this constant does not depend on the input size.
    • Consider
      • $T(n) = n^2$.
      • $T(2n) = (2n)^2 = 2^2n^2 = 2^2T(n)$
      • Doubling the input size increases the time by a constant.
    • Let's consider a different case.
      • $T(n) = 2^n$.
      • $T(2n) = 2^{2n} = 2^n2^n = 2^nT(n)$
      • This does not increase by a a constant, but by $2^n$, a value dependent on the input size.
    • This definition for efficiency is fairly good.
      • It works.
      • $n^{100}$ would be bad, but very few algorithms are on this order.
      • $n^{1+0.02log(n)}$ is ok but again very few algorithms fall into this category.
      • Algorithms tend to be
        • $n^k$ where k is a relatively small integer
        • $n^n, 2^n, n!$ all of which are very bad.
• Look at table 2.1 on page 34.
  • Note the difference between polynomial times and others.
• they point out that the final definition is also negatable.
  • Thus we have a definition for algorithms that are not efficient.
  • And can work with theory here as well.