Large and Fast, Memory

• We will discuss the principle of locality
  • As our code runs, we tend to stay in the same place.
  • Or if we used it once, it is likely that we will use it again.
  • Or if we access something, we will probably access the stuff close to it.
  • Or programs access a relatively small portion of their address space at any instant of time.
  • Two types
    • Temporal locality - related in time
    • Spacial locality - related in space.
  • Instructions have a high level of spacial locality
  • Also, depending on the code, temporal as well
  • Data exhibits both.
• Memory Overview
  • As we know, memory is arranged in a hierarchy
  • A trade off of speed vs volume vs cost

  • Type	Access Time	$ Per MB
    Disk (1997)	10-20 ms, 107ns	$0.10
    120 GB IDE (2002)	9 ms, 9x106ns	$0.0013
    73 GB SCSI (2002)	4.5 ms, 4.5x106ns	$0.0065
    DRAM (1997)	60 -120 ns	$5 - $10
    SDRAM (1997)	5-25 ns	$100 - $250
    512 MB SDRAM (2002)	5-25 ns	$0.10
    512 MB 300 MHz DDR (2002)	3 ns or less	$0.25

  • As we know clock rates are in the range of 2.5 GHz

  • 2500 x 10 6 cycles/second
    With a CPI of 1 (or even two, who cares)
    
    1 sec     109 ns
    ------- * ------ =  .4 ns/cycle
    2.5x109   sec 
    
           

  • So we need instruction or data every .4 ns ...
  • Or memory is roughly 10 times too slow.
  • Were we better off in 97?
  • PPro was 300 MHz

    1 sec     109 ns
    ------- * ------ =  3.3 ns/cycle
    300x106   sec 
    
    

  • But fast memory was around 9 ns, so the factor was 3 times
  • Remember last semester we talked about this, CPU-MEMORY gap
  • And it continues to increase.
  • 
• Cache
  • Memory on the cpu
  • Much faster than main memory
  • Much more expensive too
  • Data is copied from main memory to cache
  • If it is there when the CPU needs it, we call this a cache hit
  • If it is not, we call it a cache miss.

                 cache hits
    hit rate = --------------
               memory accesses
           

  • The miss rate is 1-hit rate
  • The time it takes to access the cache is called hit time
  • The time it takes for a miss is called the miss penalty
  • Our cpi calculation can now be modified
    • %instructions not accessing cache * CPI
    • 1+ %instructions accessing cache * (hit rate * hit time + (1-hit rate ) * miss penalty )

      34 % LW, SW
      Average CPI 1.03
      
      98% hit rate
      2 cycle hit time
      40 cycle miss penalty
      
      CPI = .66 * 1.03 + 1 +.34*(.98*2 + .02 * 40) 
          = .68 + 1 + 0.93 = 2.61
      	     

Cache Basics

• Cache is just a specialized chunk of memory
  • A linear array indexed from 0 to a maximum size.
  • But we want to keep the size smaller
    • So it is less expensive
    • So it is faster
    • So it fits on the chip
  • A Direct Mapped Cache
    • Uses the bottom bits of the address to determine where in the cache a data item is placed.
    • There is a possibility that multiple items can map to the same location.
      • 8 cache locations 000 - 111
      • 32 memory locations 00000 - 11111
      • 11111, 10111, 01111, and 00111 all map to the same location Collision
      • We need to store the rest of the address to know what is in the cache location. (called a tag)
      • We need a bit to say something is stored there.
    • memory address % bits in cache address.
    • To check to see if an item is in the cache
      • Calculate the index
      • Check to see if the top bits match the tag field
      • Check to see if the valid tag is set.
      • If everything matches, return the data in the cache entry
    • Think of this as a table with the following header fields
      Access memory 00101, 00100, 00101, 11101, 00010
      How is temporal locality addressed?
      How long does it take to access data in this cache?
      We can discount the bottom two bits, we are byte addressable, and our cache holds words.
      
      What do we do about a miss?

      • Stall the current instruction
      • Issue a fetch to the lower level of memory

      Index	V	Tag	Data