Analysis of Searching and Sorting

An Algorithm is a set of instructions that solve a problem in a finite amount of time.

BadSort(A)

 While A is not sorted
    Select two values in the array at random
    Swap those two values.

This is based on the callout box starting on page 274.
This is the main subject in algorithms and data structures (CSCI 385)
This is one of the theoretical foundations of our discipline.
How long will it take for a program to run?
- This question is hard.
- processor speeds change over time.
- Memory has an impact
- And the load on a system has an impact.
- So how can we compare things?
We have developed an abstract measurement of performance called Complexity
- Book Definition: Complexity is a measure of the effort expended by the computer in performing a computation relative to the size of the computation.
- An estimate of the number of operations that must be performed to execute a task.
We tend to measure the best, worst and average complexity of an algorithm.
We use big-o (O) notation to describe complexity.
- Given a function f(n) that describes the number of operations an algorithm may perform.
- n is the size of the input.
- The complexity $O(f(n))$ is expressed as the most significant term of the expression, minus any constant.
- if $f(n) = 5n^2+2n+2$, we say the algorithm is $O(n^2)$
- If $f(n) = \frac{1}{2}n^2+ 3n$, the algorithm is $O(n^2)$
Let's look at linear search
- ```
LinearSearch(A,key)

 i ← 0
 while i < A.size and A[i] ≠ key
       i++
 return i
```
- What is the fewest number lines 2-3 will execute?
- What is the most number of time lines 2-3 will execute?
- What will happen on average?
- This algorithm is $O(n)$ where $n$ is the number of elements in the array.
- This is really the average and worst case expectations.
What does it mean for an algorithm to be $O(n)$
- The time to execute the algorithm is directly related to the number of data elements in the input.
- As the number of elements increases, the time will increase

Look at findTimeTest.cpp

There is a little timer magic here.
The program does the following

for size from 32 to max
   Fill an array with random values
   Search the array for random values a fixed number of times.
   Print the time to search the array.

For a run for me, this produced the following table

Array Size   Time in Msec
        32             13
        64             15
       128             29
       256             57
       512            112
      1024            223
      2048            445
      4096            888
      8192           1780
     16384           3570
     32768           7148
     65536          14302
    131072          28676
    262144          57267
    524288         114575

Notice what happens to the size as I doubles.

Let's take a look at insertion sort
- ```
Sort(A) 

 for i ← 0 to A.size-2
   smallPos ← i
   for j ← i+1 to A.size-1
     if  A[smallPos] > A[j]
         smallPos ← j
   if i ≠ smallPos
     Swap(A[i], A[smallPos]);
```
- Note line 1 will run $O(n)$ times.
- but inside of this, what will happen with lines 3-5
- n-1 times the when i = 0 (j=1 to size-1)
- n-2 times the when i = 1 (j=2 to size-1)
- n-3 times the when i = 2 (j=3 to size-1)
- ...
- 1 times the when i = size-2 (j=size-1 to size-1)
- That is $ f(n) = \sum_{i=1}^{n} n-i $
- You will learn that $f(n) = \frac{(n)(n-1)}{2}$
- So $f(n) $ is $O(n^2)$.

Look at sortTimeTest.cpp

Array Size   Time in Msec     sqrt(time)
        32              2              1
        64              7              3
       128             23              5
       256             81              9
       512            297             17
      1024           1129             34
      2048           4385             66
      4096          17269            131
      8192          68522            262
     16384         272917            522

Note as the array size doubles, the square root of the time doubles.
This means the time goes up by a power of 4.
This seems to validate that the performance is $O(n^2)$