Buffer Overflow Background

• These are persistent.
• And as we have seen, still a threat.
• Sidebar 3-2 on page 135 is interesting.
  • You want to write a program that will "dial" the modem for you.
  • How will you store the phone number?
    • You call it a number, but it is really a string.
  • How many digits should you allow in your string?
  • Apparently, at one time Microsoft allowed 100
  • But the dialing program needs privilege
    • Modems are hardware, and their access is generally limited.
• The basic idea
  • Arrays are one of the fundamental data structures used in programs.
  • When using an array, the size is fixed
    • This is true even if you are using a language that supports dynamic arrays.
  • If you are not careful, you can step outside the bounds of an array.
  • When this happens, you create a flaw which can be exploited.
• Some background
• Memory
  • Memory in a computer is one giant array.
  • One of the basic principles of computer operation today is that all memory is equal.
    • There is no real difference between program memory
    • And data memory
    • And operating system memory.
  • The OS provides some protections against memory violations.
  • But most programs execute some privileged instructions which can access a wider range of memory. (more later)
  • Most users/programmers have no real idea of where their programs/data reside in memory.
• Fetch - Decode - Execute
  • The basic operation of a CPU
  • Fetch an instruction from Memory
  • Decode it, or figure out what the instruction tells you to do
  • Do what the instruction says.
• Execution of Instructions
  • Most computers (CPU) have a register called the program counter.
  • This holds the address in memory for the next instruction to be executed.
  • At the beginning of the fetch decode execute cycle, this points (or holds the address of) the next instruction to be fetched.
  • Normally this increments by one instruction each time.
  • But branches (if, while, for) cause it to change by a larger amount
    • Either negative or positive.
    • These amounts are normally stored directly in the instruction.
    • Which is stored in memory.
  • Function calls and returns are another story
    • They tend to be much larger.
    • And are often stored dynamically in memory.
• Instruction Representation
  • As you should know, all data is stored as binary strings.
  • If I look at the string with different encoding methods, the string means different things.
    • Characters are stored with ASCII codes.
    • Ints are stored as n bit two's compliment numbers.
    • Floats are stored in the IEEE 754 standard.
    • For the x86 chip, op codes are here
  • Not all bit patterns are valid instructions.
    • Usually when a CPU encounters a bad instruction it goes into a exception handler.
• Privileged vs non privileged mode.
  • Normally instructions are run in protected mode.
    • This checks for memory violations and other problems.
    • And has restricted access to hardware.
  • Special instructions and exceptions are usually run in privileged mode.
    • This normally has checks to make sure that nothing bad is accessed
    • Then will perform the restricted task
• Problems arise when data is treated as instructions and vice versa