Buffer overflow

By sending in data designed to cause a buffer overflow, it is possible to write into areas known to hold executable code and replace it with malicious code, or to selectively overwrite data pertaining to the program's state, therefore causing behavior that was not intended by the original programmer.

Buffers are widespread in operating system (OS) code, so it is possible to make attacks that perform privilege escalation and gain unlimited access to the computer's resources.

Programming languages commonly associated with buffer overflows include C and C++, which provide no built-in protection against accessing or overwriting data in any part of memory and do not automatically check that data written to an array (the built-in buffer type) is within the boundaries of that array.

In the following example expressed in C, a program has two variables which are adjacent in memory: an 8-byte-long string buffer, A, and a two-byte big-endian integer, B.

Writing data past the end of allocated memory can sometimes be detected by the operating system to generate a segmentation fault error that terminates the process.

The techniques to exploit a buffer overflow vulnerability vary by architecture, operating system, and memory region.

The Metasploit Project, for example, maintains a database of suitable opcodes, though it lists only those found in the Windows operating system.

Exploitation is performed by corrupting this data in specific ways to cause the application to overwrite internal structures such as linked list pointers.

Manipulations could include conversion to upper or lower case, removal of metacharacters and filtering out of non-alphanumeric strings.

These factors include null bytes in addresses, variability in the location of shellcode, differences between environments, and various counter-measures in operation.

[7] It solves the problem of finding the exact address of the buffer by effectively increasing the size of the target area.

[8] Because of the popularity of this technique, many vendors of intrusion prevention systems will search for this pattern of no-op machine instructions in an attempt to detect shellcode in use.

Any instruction that does not corrupt the machine state to a point where the shellcode will not run can be used in place of the hardware assisted no-op.

As a result, it has become common practice for exploit writers to compose the no-op sled with randomly chosen instructions which will have no real effect on the shellcode execution.

[10] An incorrect guess will usually result in the target program crashing and could alert the system administrator to the attacker's activities.

The traditional solution is to find an unintentional instance of a suitable opcode at a fixed location somewhere within the program memory.

Assembly, C, and C++ are popular programming languages that are vulnerable to buffer overflow in part because they allow direct access to memory and are not strongly typed.

Examples of such languages include Ada, Eiffel, Lisp, Modula-2, Smalltalk, OCaml and such C-derivatives as Cyclone, Rust and D. The Java and .NET Framework bytecode environments also require bounds checking on all arrays.

Static code analysis can remove many dynamic bound and type checks, but poor implementations and awkward cases can significantly decrease performance.

Software engineers should carefully consider the tradeoffs of safety versus performance costs when deciding which language and compiler setting to use.

Microsoft's implementation of Data Execution Prevention (DEP) mode explicitly protects the pointer to the Structured Exception Handler (SEH) from being overwritten.

CHERI (Capability Hardware Enhanced RISC Instructions) is a computer processor technology designed to improve security.

It operates at a hardware level by providing a hardware-enforced type (a CHERI capability) that authorises access to memory.

Address space layout randomization (ASLR) is a computer security feature that involves arranging the positions of key data areas, usually including the base of the executable and position of libraries, heap, and stack, randomly in a process' address space.

Randomization of the virtual memory addresses at which functions and variables can be found can make exploitation of a buffer overflow more difficult, but not impossible.

The use of deep packet inspection (DPI) can detect, at the network perimeter, very basic remote attempts to exploit buffer overflows by use of attack signatures and heuristics.

It was formerly used in situations in which a long series of No-Operation instructions (known as a NOP-sled) was detected and the location of the exploit's payload was slightly variable.

Buffer overflows were understood and partially publicly documented as early as 1972, when the Computer Security Technology Planning Study laid out the technique: "The code performing this function does not check the source and destination addresses properly, permitting portions of the monitor to be overlaid by the user.

[42] Later, in 1995, Thomas Lopatic independently rediscovered the buffer overflow and published his findings on the Bugtraq security mailing list.

[43] A year later, in 1996, Elias Levy (also known as Aleph One) published in Phrack magazine the paper "Smashing the Stack for Fun and Profit",[44] a step-by-step introduction to exploiting stack-based buffer overflow vulnerabilities.