Precision and Gap

• This is a great place for us to look at a few problems with the representation of fractions.
• Precision is the amount of space allocated for storing numbers.
  • Measured in bits.
  • This is both hardware and software dependent.
  • The hardware has a fixed size.
  • But we can write new software to take the place of hardware.
• * precision
  • Normally means a single word is used to store the number
  • double precision: - two words
  • quad precision four words
• So let's try something in 8 bit precision
  • .1 in base 10
  • Convert this to base 2

  • .1 x 2 = .2   record .0
    .2 x 2 = .4   record .00
    .4 x 2 = .8   record .000
    .8 x 2 = 1.6   record .0001
    .6 x 2 = 1.2   record .00011
    .2 x 2 = .4    record  .000110   
    note this will repeat
            
    .10 = .00011001100112
            

  • Just like $1/3 = .\overline{3}$
• So in 8 bits .1 = $.00011001_2$
  • This has a decimal value of 0.09765625
  • But what about the $.000000001\overline{0011}$?
  • This is what we call roundoff error.
• Note there is a number "before" $.00011001_2$
  • Is this true in general of rational numbers?
  • What would this number be?
  • $.00011000_2 = .09375$
  • Why?
  • What would the next number be?
  • $.00011010_2 = 0.1015625$
• .1 falls into the gap between $.00011001_2$ and $.00011010_2$
  • The gap represents an interval of numbers that are "missing" in a representation.
  • This changes based on the place we are looking at in the number line.
  • We will see more of this later.