Polynomial Time as a Definition of Efficiency

Objectives

We would like to :

Discuss why polynomial time is equated with efficiency
Discuss what we measure for performance
Formally define O(f(n)).

Notes

This continues chapter 2 of Kleinberg and Tardos.
When we analyze an algorithm, what is it we would like to count?
- In architecture we learned
  - time = instruction count × cycles per instruction × seconds per cycle.
  - The time per instruction is fixed by the machine and changes as the machine changes
  - IE gets faster as new machines are introduced.
- But does the instruction count change?
  - Actually we know this does vary by machine
  - Consider the c/c++ code
  - a = b + c;
  - In intel assembly:
  - ```
  mov eax,[b]
  add eax,[c]
  move [a], eax 
```
- but in mips assembly
```
  lw $t1, b 
  lw $t2, c
  add  $t2, $t2, $t1 
  sw $t22, a
```
- A difference of one instruction, but over time ...
- And some higher level operations are much worse.

Let's discuss the number of instructions for

const int SIZE{some number}
int a[SIZE],b[SIZE],c[SIZE];
 
for(i = 0; i < SIZE; ++i) {
   a[i] = b[i] + c[i] 
}

In intel assembly:


    mov ecx, 1            ; i = 1
    mov ebx, SIZE         ; loop bound

loop:
    cmp ecx, ebx
    jge done              ; if i >= SIZE, exit loop

    mov eax, [b + ecx*4]  ; eax = b[i]
    add eax, [c + ecx*4]  ; eax = b[i] + c[i]
    mov [a + ecx*4], eax  ; a[i] = eax

    inc ecx               ; i++
    jmp loop

done:

The loop takes 7 instructions.
And there are two instructions for setup
The performance is 7*SIZE + 2

The loop in mips assembly


    li   $t0, 1          # i = 1
    li   $t1, SIZE       # loop bound

loop:
    bge  $t0, $t1, done  # if i >= SIZE, exit loop

    sll  $t2, $t0, 2     # offset = i * 4 (since word = 4 bytes)

    lw   $t3, b($t2)     # load b[i]
    lw   $t4, c($t2)     # load c[i]
    add  $t5, $t3, $t4   # t5 = b[i] + c[i]
    sw   $t5, a($t2)     # store into a[i]

    addi $t0, $t0, 1     # i++
    j    loop 
done:

(Again, thanks to chatGPT)

takes 8 instructions.
There are also two instructions for setup.
The performance is 8*SIZE + 2

Does this mean that the algorithm in intel assembly is better than the algorithm in mips assembly?

What if I implemented the DAN architecture that had an instruction

```
add mem, mem, mem.  
```

My loop might be something like:


    li   $t0, 1          # i = 1
    li   $t1, SIZE         # loop bound

loop:
    bge  $t0, $t1, done  # if i >= SIZE, exit loop

    sll  $t2, $t0, 2     # offset = i * 4 (since word = 4 bytes)

    add  a($t2), b($t2), c($t2)

    addi $t0, $t0, 1     # i++
    j    loop

(My own brain!)

My time would be 5*SIZE + 2
Clearly the "algorithm" is better on the DAN computer?

By the way, we probably should add a 1 for the extra bge or 2 for the cmp, jge instructions that execute one more time then the loop.
Or is this just implementation noise?
Side note: what does c1_arch×SIZE + c2_arch represent in terms of the code?
Let's take a look at some functions.
- Traditionally code falls into
  - Log based: or c×log₂(n)
    - In this case, c must be constant
  - Polynomial: a_inⁱ + a_i-1n^i-1 + ... + a₁n + a₀.
    - a_i and i are constant.
  - Combination of log and polynomial: c×n^dlog₂n
    - c and d are constant.
  - Constant to n: cⁿ, we shall see this is a different beast.
  - Factorial: n!, again this is a different beast.
- The book on page 34 has a comparison chart.
  - Looking at the first four columns...
    - Do the non leading degree of the polynomial terms matter?
    - Look at for example n³, n³+ n²
      - n = 10,000 : 12 days vs 12 days and 2 minutes
      - Compare this to 2ⁿ or n!: very long
    - How about if we have a leading coefficient?
      - n = 10,000: 12 days vs 24, 36, 48 days
      - Compare this to 2ⁿ or n! : vary long
    - How about n log₂n?
      - Is this radically different from n or n²?
  - Use desmos to explore these.
a_inⁱ + a_i-1n^i-1 + ... + a₁n + a₀.
- How important is the a_i term?
  - Isn't it mostly just an indication of the number of instructions in the loop in a particular assembly language?
  - Isn't it really not a major contributor to time (nⁱ dominates).
- How important are a_i-1n^i-1 + ... + a₁n + a₀?
  - We will see these represent setup and completion of the main "loop".
  - But from above, they contribute, but not as much as the leading term.
- So in the end, we only look at the degree of the polynomial, or ...
In the end, we don't care what instructions, we only care that the instructions are high level.
Think about selection sort
- ```
SELECTION-SORT(A)
   Input: A an array of items with the comparison operator
   Output: the array A ordered by the comparison operator

 current ← 0
 while current < A.size
    small ← current
    for next ← current +1 to A.size
       if A[next] < A[small]
           small ← next
    if small ≠ current
       swap(A[small], A[current])
    current ← current + 1
 
```
- What does this algorithm do?
- How does it do it?
- Will it work?
- What is the performance of this algorithm?
  - How many times will the loop in line 2 execute?
  - How many times will the loop in line 4 execute?
- Are all of the instructions basic?
- Can you come up with the exact polynomial for this algorithm? (ie true instruction count?)
  - On linux you can run perf to determine the number of instructions executed.
  - I implemented the sort in ssort.cpp.
  - I ran it ten times for 1,000 elements to 10,000 elements, by 1,000.
The book states: An algorithm is efficient if it has polynomial running time.
O(g(n)) = {f(n): there exist positive constants c and n₀ such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n₀}