Memory Corruption and Code Injection

Memory-safety bugs violate an object’s bounds or lifetime. They can expose secrets, corrupt data, or let an attacker redirect a program’s execution.

Hijacking and Injection

Control-flow hijacking redirects a program’s execution, usually by corrupting a return address or a function pointer. Code injection places the attacker’s own instructions in the program and arranges to run them. A stack buffer overflow often does both at once. Hijacking and injection are related actions rather than separate categories.

Running code in another process gives the attacker that process’s access. This is worth doing when the program runs with more privilege than the attacker has, when it runs on a machine the attacker cannot log in to, or when it holds secrets the attacker cannot otherwise read. A setuid program runs with its owner’s privileges, so a setuid-root program can give an attacker root. Making a network service run attacker code is remote code execution.

Behind each of these is a program that made an assumption about its input without checking it. A vulnerability is a weakness that can be exploited; an exploit is the input or program that takes advantage of it. The same bug can crash one build and allow code execution on another, because exploitability depends on the surrounding code, the memory layout, and the defenses in place. Encryption and signatures do not make a vulnerable parser memory-safe: the receiving program still has to parse the input, sometimes before it can verify it.

Memory Layout and the Stack

A process runs in its own virtual address space, divided into regions: the read-only text segment of instructions, the data and BSS segments of global variables, the heap for dynamically allocated memory, and a stack for each thread. The program writes to the stack, the heap, and its globals, so all three can be overflow targets. The text segment is read-only.

The stack supports function calls. In the conventional x86 layout used here, each call gets a stack frame holding its stack-passed arguments, saved registers, local variables, and a return address that the processor jumps to when the function returns. Arguments and locals can instead live in registers, and return-address handling varies by architecture. In this layout a local array sits at a lower address than the return address, and arrays fill toward higher addresses, so a write that runs off the end of a local array moves toward the saved return address.

Stack Buffer Overflows

A buffer overflow writes more data into an array than it can hold, overwriting whatever is next in memory. Writing past an array is undefined behavior in C: the processor checks page permissions, not array bounds, so the write usually succeeds silently and the problem surfaces later. When the overflow reaches the return address, the function’s return can send the processor to an invalid address (a crash, which is a denial of service), to existing code, or to attacker-supplied instructions in the buffer.

An overflow that changes a nearby decision variable rather than the return address is a data-only attack: the program runs its own instructions in order and still does the wrong thing. Whether a given overflow reaches a particular variable or the return address depends on how the compiler laid out the frame.

gets() cannot be used safely because it is never told the size of its buffer. The counted functions take a length, but each counts something different: strncpy writes n bytes, padding short strings with null bytes but possibly leaving longer strings unterminated, strncat limits the bytes appended rather than the destination size, and snprintf counts the terminating null within its limit. They prevent an overflow only when the programmer passes the right size, leaves room for the terminating null, and handles truncation. That size has to come from the destination’s capacity, not from sizeof applied to a pointer, which yields only the pointer’s width. The flaw is a missing or wrong size check, not any one function name.

Shellcode is machine code an attacker places in memory to run after a hijack, written to avoid bytes the input path rejects, such as null bytes, and to fit whatever space the bug makes available. A NOP sled is a run of do-nothing instructions before the shellcode that widens the range of addresses a jump can land on. Neither helps once the target memory is non-executable.

Reading Past a Buffer

An out-of-bounds read accesses memory beyond an object’s bounds. It can disclose secrets or pointers when the program exposes the result, or it can crash the program. A leaked pointer reveals the base of a randomized region and undermines address randomization for that region. Reading past a buffer is a vulnerability on its own, separate from the write overflows above.

Heap Corruption and Lifetime Errors

Heap memory is handed out by an allocator that keeps bookkeeping for each block. Traditional allocators often store that metadata next to the user data; others keep it separately. An overflow out of one block can corrupt a neighboring object or the allocator’s metadata, and in older allocators a forged pointer could produce an arbitrary write to an address of the attacker’s choosing. Overwriting a function pointer or a C++ vtable pointer redirects the next call.

Two errors violate an object’s lifetime rather than its bounds. A use-after-free uses a pointer after the memory is freed; if the attacker arranges for the freed block to be reused for their own data, the stale pointer leads into it. A double-free frees the same block twice, which can corrupt the allocator’s bookkeeping or make a hardened allocator abort. These are temporal errors, as opposed to the spatial errors of overflows, and they are hard to find because the symptom appears far from the cause. Type confusion uses an object as a type it is not, so the code reads its fields with the wrong layout and may treat ordinary data as a pointer.

Integer Errors

Fixed-width integer types have limited ranges. Unsigned overflow wraps in a way C defines; signed overflow is undefined behavior, so a compiler may assume it never happens and drop a check that depends on it. An integer error becomes a memory error when a wrapped size produces an allocation smaller than the code believes, followed by writes that overrun it. Checking that a size calculation fits before performing it, and rejecting a negative length before it is converted to an unsigned size, closes that path. A signed-to-unsigned mismatch lets a negative length pass a signed upper-bound check and then become a huge value when passed as an unsigned size to a copy function. Narrow protocol fields and the 32-bit default int keep these bugs reachable even on 64-bit systems.

An off-by-one error reads or writes one position past a boundary. A write one position past can corrupt a saved value or the low byte of a pointer, and it can be the start of a larger overflow.

Defenses

No single defense closes the hole. Each answers a specific attack and leaves something open, so they are deployed in layers.

The first layer prevents and finds bugs before release:

Auditing, static analysis, and compiler warnings catch missing checks and unsafe calls.
Sanitizers add runtime checks that report memory and arithmetic errors in the code exercised during testing.
Fuzzing feeds malformed input and watches for failures; coverage-guided fuzzers steer input toward new code.
Memory-safe languages such as Rust and Go check bounds and manage lifetimes, so an invalid access is caught rather than allowed to corrupt memory; the guarantee stops at unsafe code and native libraries.

When a bug survives, the next layer makes it harder to exploit. Non-executable memory stops the processor from running injected code and forces attackers to reuse existing code. NX is the hardware page permission, DEP is the operating-system protection built on it, and W^X is the policy that no page is writable and executable at the same time. They are related, not three names for one thing.

Non-executable memory does not stop code reuse. A return-to-libc attack redirects execution to an existing library function, and return-oriented programming (ROP) generalizes it by chaining short instruction sequences called gadgets, each ending in a return, with the chain supplying successive gadget addresses and any operands they need. A sufficient set of gadgets can perform arbitrary computation without injecting any code.

Three more defenses raise the cost of a surviving bug:

Address space layout randomization (ASLR) places code and data at unpredictable addresses; its strength is its entropy, the amount of randomness in that placement, and a leaked pointer can expose a randomized region’s base and undo it for that region.
Stack canaries place a secret value between local buffers and the return address and check it before returning, catching a contiguous overflow but not a heap bug, a leaked canary, or a data-only attack.
Hardened allocators check and protect their metadata and delay reuse of freed blocks.

Hardware and compiler checks detect invalid control transfers, pointer tampering, and invalid memory accesses:

Control-flow integrity (CFI) checks that an indirect transfer goes to an allowed target, though an attacker-chosen destination that stays within the allowed set still passes, and forward-edge checks do not protect returns.
Intel CET combines a hardware shadow stack, which compares a protected copy of each return address with the one on the normal stack, with indirect branch tracking, which checks that indirect calls and jumps land at marked targets.
ARM Pointer Authentication (PAC) authenticates the pointers that software signs and checks, so tampering with them is detected; pointers it is not applied to stay unprotected.
ARM Memory Tagging Extension (MTE) tags memory and pointers so an enabled check faults on a mismatch, catching many overflows and use-after-free; an access that stays within a granule, a small fixed-size block of memory sharing one tag, or that happens to match the tag can escape it, and a fault may be reported immediately or asynchronously.

Defense in depth combines these independent measures so an exploit must defeat every one on its path, which pushes an attacker toward rarer and less reliable conditions. A data-only attack bypasses most of them, so a memory-safe language is the deeper fix. Rowhammer is a reminder that these defenses assume memory holds what was written to it, which repeated access to dense DRAM can violate.

What You Don’t Need to Study

Incident names, CVE numbers, dates, versions, and which incident demonstrated which bug, including the OpenSSL, HP Poly, SMBGhost, FORCEDENTRY, and sudo cases. Know the vulnerability mechanisms described above and recognize them from a description.
How to construct shellcode, NOP-sled payloads, ROP chains, or forged allocator metadata. Know what each technique accomplishes and which defenses obstruct it.
The names or construction details of heap grooming and heap spraying. Know that reuse of freed memory can place attacker-controlled data behind a stale pointer.
Format-string vulnerabilities or the %n write primitive, which are covered in recitation.
Specific instructions, register names, exact offsets, or a memorized stack diagram. Know how a write past a local buffer can reach nearby data or a saved return address in the layout used here.
The exact contract of each counted string function. Know that a size check has to use the destination’s capacity and leave room for the terminating null.
The internal field names or algorithms of a heap allocator. Know that it keeps metadata an overflow can corrupt, not its exact layout.
The names of particular fuzzers, sanitizers, static analyzers, or compiler options.
Entropy bit counts, tag widths, granule sizes, collision probabilities, authentication-code calculations, operating-system versions, overhead percentages, deployment counts, or the specific processors and phones that ship each hardware defense. Know what each defense does and what it leaves open.
How Intel CET’s indirect branch tracking works, how ARM Pointer Authentication signs and verifies pointers, and how ARM Memory Tagging Extension assigns and checks tags.
How Rowhammer works internally or which memory modules were tested. Know that hardware can be made to flip bits and that the software defenses above it do not fully prevent it.