Finish of Intro:

• The Power Wall:
  • Until recently, we have increased clock speed regularly
  • See figure 1.15, page 39
  • For our current technology:

    Power = Capacitive load × Voltage2 × Frequency switched.
    	 

  • Frequency switched is a function of the clock rate.
  • Capacitive load is a function of
    • the number of transistors connected to a output
    • The underlying technology
  • Voltage is the other portion of the equation
    • We have been able to drop the voltage as we increase the speed
    • But we have hit a wall.
    • They tell us that currently 40% of the power is lost (or leaks)
    • We can't efficiently cool the chips fast enough to increase the speed.
  • Hence we have gone to multi-core machines.
• Parallel Hardware.
  • Flynn's Taxonomy

    	Single Instruction	Multiple Instruction
    Single Data	SISD	MISD
    Multiple Data	SIMD	MIMD

  • Also SPMD, but not really hardware.
  • Also MPMD, but not really hardware.
• Issues with Parallel Hardware
  • SISD is easy to program,
    • Clock speed increases mean your program gets faster.
  • SIMD is easy to program but expensive to build (until recently)
    • But increase the number of processors only helps when the problem can be broken up nicely.
    • Some programs are inherently sequential.
    • Some programs don't fit on a SIMD nicely
    • Programmers mostly don't need a new way of thinking about programming when SIMD works.
    • Scaling works up to data sizes.
  • MIMD is hard to program.
    • See notes above.
    • We need a completely new way of thinking about programming.
    • There are new types of issues:
      • Sharing memory
      • Deadlock
      • Race Conditions
      • Scheduling tasks
      • Synchronization
      • Load Balancing
      • Scaling to more processors is an issue.
  • All parallel processing is subject to Amdhal's law
    Two statements:

    	           Execution Time affected by improvement
    Execution Time = ----------------------------------------- + Execution time not affected by improvement
                               Amount of Improvement
    
               Parallel Part 
    time = -----------------------  + Sequential Part
            number of processors
    	

    • Ken Batcher "A supercomputer is a device for turning compute-bound problems into I/O-bound problems."
  • Benchmarks:
    • Is there a good way to compare machines in general?
    • Histoically bad benchmarks
      • Clock Speed
      • MIPS - just clock speed.
      • A closed vender benchmark.
    • SPEC - Standard Performance Evaluation Corporation
      • Industry wide consortium
      • Establishes and maintains Real world benchmarks.
      • These measure many things (look here)
      • Information, including compiler options are provided.
      • Results are compared against a base machine, to provide a common number for comparison.
    • Several tests are combined (via geometric mean) to find a Single number.
    • Look at page 49 and on line.