pk.org: Computer Security/Lecture Notes

Memory Corruption and Code Injection

Terms we covered

Paul Krzyzanowski – 2026-10-17

Attacks and Goals

Control-flow hijacking
Redirecting a program’s execution by corrupting a return address or function pointer.
Code injection
Placing attacker-supplied instructions in a process and causing them to run.
Shellcode
Machine code supplied as a payload, often to start a command shell.
Remote code execution
Exploiting a vulnerability to run attacker-chosen code on a remote system.
setuid program
A program that runs with its owner’s privileges rather than the caller’s.
Vulnerability
A weakness that can be exploited to perform an unauthorized action.
Exploit
An input or program that takes advantage of a vulnerability.

Memory Layout

Virtual address space
The range of virtual addresses available to a process.
Stack
The region holding the active function calls of a thread.
Stack frame
The storage set aside for one function call.
Return address
The saved location a function returns to when it finishes.
Heap
The region holding memory allocated while a program runs.

Overflows

Buffer overflow
Writing more data into a buffer than it can hold.
Undefined behavior
An operation the C standard leaves unconstrained, such as an out-of-bounds access.
Data-only attack
Changing a program’s data to alter its behavior without redirecting control.
Out-of-bounds read
Reading past a buffer, which can disclose adjacent memory.
Off-by-one error
A counting or boundary error that differs from the correct value by one.
NOP sled
A run of do-nothing instructions that widens the landing area for a jump.

Heap and Lifetime Errors

Memory allocator
The library that hands out and reclaims heap memory and tracks its blocks.
Arbitrary write
An attacker’s ability to write a chosen value to a chosen address.
vtable
A C++ object’s table of pointers to its virtual functions.
Use-after-free
Using a pointer after the memory it refers to has been freed.
Double-free
Freeing a memory allocation that has already been freed.
Spatial error
A memory error that accesses outside an object’s bounds.
Temporal error
A memory error that accesses an object outside its lifetime.
Type confusion
Treating an object in memory as a type it is not.

Integer Errors

Integer overflow
An arithmetic result that exceeds the range of its fixed-width type.
Signed-to-unsigned mismatch
An error caused by converting a signed value to unsigned, potentially changing comparisons or sizes.

Defenses

Sanitizer
A compiler option that adds runtime checks to catch memory and arithmetic errors.
Fuzzing
Feeding a program many malformed inputs to find crashes and other failures.
Memory-safe language
A language that checks bounds and manages object lifetimes for the programmer.
Non-executable memory
Marking data pages so the processor refuses to run code from them.
Return-to-libc
A code-reuse attack that redirects execution to an existing library function.
Return-oriented programming (ROP)
Chaining short existing instruction sequences, called gadgets, to compute without injecting code.
Gadget
A short sequence of existing instructions ending in a return, used in ROP.
Address space layout randomization (ASLR)
Loading code and data at unpredictable addresses to hide their locations.
Entropy
The amount of randomness in a placement, which sets how hard it is to guess.
Stack canary
A guard value between local buffers and saved control information, checked before returning.
Hardened allocator
An allocator that checks and protects its metadata and delays reuse of freed memory.
Control-flow integrity (CFI)
Checking at runtime that a control transfer goes to an allowed target.
Shadow stack
A protected copy of return addresses used to detect a tampered return address.
Pointer authentication (PAC)
Signing a pointer with a keyed code so later tampering is detected.
Memory tagging
Detecting invalid memory accesses by comparing tags attached to pointers and memory.
Defense in depth
Combining multiple defenses to reduce reliance on any single protection.
Rowhammer
A hardware attack that flips memory bits by repeatedly accessing nearby rows.