Stack Buffer Overflow (Vanilla)

Overview

Stack buffer overflows allow attackers to overwrite the return address on the stack, redirecting execution to attacker-controlled shellcode. This was the foundational memory corruption technique that defined exploitation for over a decade.

The Attack

How It Worked

When a program copies user input to a fixed-size stack buffer without bounds checking, attackers can overflow the buffer and overwrite adjacent memory—including the saved return address. By pointing the return address at shellcode (also placed in the buffer), attackers gain arbitrary code execution.

Stack Layout (vulnerable):
+------------------+
| Local Variables  |  <- Buffer here
+------------------+
| Saved EBP        |  <- Overwritten
+------------------+
| Return Address   |  <- Overwritten to point at shellcode
+------------------+
| Function Args    |
+------------------+

Why It Worked

• No memory protections — Stack was executable by default
• No address randomization — Shellcode addresses were predictable across reboots
• Unsafe C functions — strcpy(), gets(), sprintf() had no bounds checking
• No compiler mitigations — Compilers didn’t add protective measures

Notable Campaigns

• Morris Worm (1988) — fingerd buffer overflow; first major internet worm, infected ~6,000 machines (10% of the internet)
• Code Red (2001) — IIS idq.dll buffer overflow; 359,000 hosts infected in 14 hours
• SQL Slammer (2003) — SQL Server resolution service overflow; fastest spreading worm in history, doubled every 8.5 seconds

How The Defenses Work

Stack Canaries (1998)

The compiler inserts a random value (“canary”) between local variables and the saved return address. Before the function returns, it checks if the canary was modified. If the canary doesn’t match, the program terminates immediately.

Stack Layout (protected):
+------------------+
| Local Variables  |  <- Buffer here
+------------------+
| CANARY           |  <- Random value, checked before return
+------------------+
| Saved EBP        |
+------------------+
| Return Address   |
+------------------+

An attacker overflowing the buffer will corrupt the canary before reaching the return address, triggering detection.

DEP/NX (2004)

Data Execution Prevention marks memory regions as non-executable. The stack, heap, and data sections are marked NX (No eXecute). Even if an attacker overwrites the return address to point at their shellcode, the CPU refuses to execute code from these regions.

The attack chain breaks at: “jump to shellcode” → CPU raises exception.

ASLR (2007)

Address Space Layout Randomization randomizes the base addresses of the stack, heap, libraries, and executable. Attackers can no longer hardcode addresses because they change every execution.

The attack chain breaks at: “calculate shellcode address” → address is unpredictable.

Current State

Status: 🔴 Dead

Vanilla stack overflows are effectively neutralized on modern systems. DEP and ASLR are enabled by default on Windows (Vista+), Linux (kernel 2.6.12+), and macOS.

Exploitation now requires chaining multiple techniques:

1. Information leak to bypass ASLR (disclose a pointer, calculate base addresses)
2. ROP chain to bypass DEP (use existing code gadgets instead of injected shellcode)
3. Additional techniques for CFI bypass on hardened targets

Still occasionally seen targeting:

• Legacy/embedded systems without ASLR
• IoT devices running old kernels
• Unpatched servers running ancient software

Detection Guidance

Modern systems will crash (canary violation) or throw access violations (DEP) rather than allow exploitation. Monitor for:

• Repeated crashes in the same application
• Crash dumps showing stack corruption patterns
• Exploit kit traffic targeting known CVEs in legacy software
• Process termination with STATUS_STACK_BUFFER_OVERRUN (Windows)