Mathematical Analysis of Non recursive Algorithms

• We should analyze the performance of some easy algorithms.
• We will do this the rest of the semester.
• Example 1: Find the largest element in a list of n numbers.

  • MAX_ELEMENT(A)
    
     maxval ← A[0]
     for i ← 1 to n-1 do
          if A[i] > maxval
               maxval ← A[i]
     return maxval
    
    	 

  • Does this terminate?
  • Does it work? Why?
  • What is n in this case?
  • Should the basic operation be line 3 or line 4?
    • We really should consider line 3
  • Next find a way to express the number of times line 3 is executed in terms of n
  • This will be the same for best, worst and average, why?
    • C(n) = 1 + 1 + 1 + ... + 1 (n-1 times)
    • C(n) = Σ_i=1^n-1 1
    • C(n) = n-1 (see p 476)
    • C(n) ∈ Θ(n)
• He presents a general plan for this:
  1. Decide on a parameter (or parameters) to indicate input size
  2. Identify the basic operation. (It is generally in the innermost lop)
  3. Check whether the number of times the basic operation is executed depends only on the size of an input. If it also depends on some additional property, the worst, average and best case times need to be investigated separately.
  4. Set up a sum expressing the number of times the basic operation is executed.
  5. Simplify the sum to find the complexity class or order of growth.
• Appendix A is really useful here.
  • Σ_i=l^u cf(i) = cΣ_i=l^u f(i)
  • Σ_i=l^u (f₁(i) +/-f₂(i)) = Σ_i=l^u f₁(i) +/- Σ_i=l^u f₂(i)
  • Σ_i=0ⁿ i = Σ_i=1ⁿi = 1/2(n)(n+1)
• Another Algorithm

  • UNIQUE_ELEMENTS(A)
    
     for i ← 0 to n-2 do
         for j ← i+1  to  n-1 do
             if A[i] = A[j]
                 return false
     return true
    
            

  • Does this terminate?
  • Does this work?
  • What is the measure of the input size?
  • What is the critical operation?
  • Note here that line 4 might let us exit the algorithm early
    • This means that the best/worst/average cases could be different.
  • What is the best case?
  • What is the worst case?
    • The Outer loop is Σ_i=0^n-2 f(i)
    • The inner loop is Σ_j=i+1^n-1 g(j)

    • g(i) = 1
      Σj=i+1n-1 1 = n-1-(i+1) + 1
                = n-i-1
      
      Σi=0n-2 (n-i-1) = Σi=0n-2 (n-1)  - Σi=0n-2 i
                     = (n-1)  Σi=0n-2 1 -  Σi=0n-2 i
      	       = (n-1) (n-2 -0 + 1 ) + (n-2)(n-1) * 1/2
      	       = (n-1)(n-1) - (n-2)(n-1)/2
      	       = (n-1)/2 [2(n-1)- (n-2)]
      	       = (n-1)/2 [2n -n -2 +2]
      	       = (n-1)/2 [n]
      	       ∈ Θ(n2)
      	     

    • Another useful way to look at this is:
      • The first time the inner loop is run, it will execute 1 to n-1 or n-1 times.
      • The second time it will execute 2 to n-1 or n-2 times.
      • ...
      • T(n) = n-1 + n-2 n-3 + ... + n + 1
      • T(n) = Σ_i=1^n-2 n-i
    • This type of loop always seems to confuse some people, they don't want it to be Θ(n²), BUT IT IS!
    • Average case for this algorithm is much more complex.
      • It depends quite a lot on the data
      • And the distribution of the data.
        • Sorted vs not sorted.
        • Homogeneous vs mostly unique.
      • If we assume mostly unique data, randomly distributed it will be very close to the previous example, and O(n²)
      • If we assume sorted, it will be O(n) (WHY?)
      • If we assume mostly homogeneous it will be O(1)
• A cautionary tale:
  • Given a list of numbers, compute each to the 100,000 power.
  • Assume power(n, p) ∈ O(lg(p))

  • BIG_POWER(A)
    
     for i ← 0  to n-1 do
         A[i] ← power(A[i], 100,000)
     return
    
    	

  • Does this algorithm terminate?
  • Does it work?
  • What is the performance?
• Another Example: Matrix Multiplication
  • Assume two n x n matrices, A and B.
  • Compute C = AxB

  • MATRIX_MULTIPLY(A,B)
    
     for i ← 0  to n-1  do
         for j ← 0  to n-1  do
              C[i,j] ← 0
              for k ← 0   to  n-1  do  
                  C[[i,j] ←  C[i,j] + A[i,k] * B[k,j]
     return C
    
    	 

  • Will this terminate?
  • Will it work?
  • What is the performance?
  • He points out that we can count either multiples, adds or both.
  • But the asymptotic performance is the same for both.
• A final example: Integer Power
  • Compute xⁿ, n ∈ Z⁺

  • POWER(x,n)
    
     result ← 1
      while n > 0  do
          if  n % 2 != 0 
            result ← result * x 
         x ← x * x  
         n ← n / 2
     return  result
    
    	 

  • Does this terminate?
  • Does this work?
    • note x^ax^b = x^a+b
  • Do we need to trace it?
  • Performance
    • Input size?
    • Critical operation?