How Memory Sanitizers Catch Heap Corruption in C

Memory sanitizers are dynamic analysis tools designed to detect memory safety violations—such as buffer overflows, use-after-free errors, and invalid frees—at runtime. This article explains the fundamentals of memory sanitization in the C programming language, focusing on AddressSanitizer (ASan). It details how sanitizers modify compiled code through compiler-level binary instrumentation, manage shadow memory, and intercept heap allocations to detect and report heap corruption immediately when it occurs.

What Are Memory Sanitizers?

In standard C, memory management is manual. Functions like malloc(), calloc(), and free() allocate and release chunks on the heap, but the runtime provides no native bounds checking or lifecycle tracking. As a result, programming errors such as writing past the boundary of an allocated block (heap buffer overflow) or accessing memory that has already been deallocated (use-after-free) often go unnoticed until undefined behavior crashes the program or introduces a security vulnerability.

Memory sanitizers bridge this gap. Rather than relying on purely static analysis, tools like AddressSanitizer combine compile-time instrumentation with a dedicated runtime library to actively monitor memory access patterns during execution, failing fast the moment illegal access occurs.

How Sanitizers Rewrite Code and Binaries

Memory sanitizers do not typically disassemble and modify existing compiled machine code in a post-processing step; instead, they operate as optimization and transformation passes during compilation (such as inside LLVM/Clang or GCC). The compiler rewrites the program's Intermediate Representation (IR) before emitting machine code, fundamentally altering how heap operations and memory accesses are executed.

This transformation relies on three core mechanisms:

1. Shadow Memory Architecture

The sanitizer maps the entire virtual address space of the application to a dedicated region called "shadow memory." A fixed scale and offset are used to map every 8 bytes of regular application memory to 1 byte of shadow memory:

\[\text{ShadowAddress} = (\text{AppAddress} \gg 3) + \text{Offset}\]

The value stored in that single shadow byte indicates the state of the corresponding 8 application bytes:

  • 0: All 8 bytes are unpoisoned (valid and addressable).
  • \(k\) (where \(1 \le k \le 7\)): The first \(k\) bytes are valid; the remaining \(8 - k\) bytes are poisoned (inaccessible).
  • Negative value: All 8 bytes are poisoned due to a specific reason (e.g., heap redzone, freed memory, stack redzone).

2. Redzones and Heap Allocation Interception

The sanitizer's runtime library overrides the default standard C library allocators (malloc, calloc, realloc, free).

When an application calls malloc(size):

  1. The custom allocator requests additional memory to surround the requested buffer with "redzones"—inaccessible padding placed immediately before and after the allocated block.
  2. The memory corresponding to the usable buffer is marked as unpoisoned in the shadow memory.
  3. The memory corresponding to the redzones is explicitly poisoned in the shadow memory.
  4. The allocator returns a pointer to the start of the valid buffer, hiding the redzones from the application logic.

When the application calls free(ptr):

  1. The memory region is not immediately returned to the operating system or made available for reuse.
  2. The entire region is marked as poisoned (specifically tagged as "freed memory") in the shadow memory.
  3. The chunk is placed into a quarantine queue (a FIFO buffer), preventing immediate reallocation so that dangling pointers cannot inadvertently read newly assigned data.

3. Memory Access Instrumentation

For every memory read (load) and write (store) instruction in the C source code, the compiler inserts an inline validation check prior to the access.

For instance, an instruction like:

*address = value;

is rewritten at the machine-instruction level to check the shadow memory first:

byte *shadow_address = (address >> 3) + Offset;
if (*shadow_address != 0) {
    if (IsPoisoned(shadow_address, address)) {
        ReportCrashAndTerminate(address);
    }
}
*address = value;

Because the address translation requires only a bit-shift and an addition, and the validation requires a single byte load and comparison, this inline check executes with minimal performance overhead relative to interpreted memory checkers.

Catching Heap Corruption in Practice

With instrumentation and shadow memory in place, detecting heap corruption becomes deterministic:

  • Heap Buffer Overflow: If an array index exceeds its allocated bounds, the pointer increments into the post-allocation redzone. When the instrumented load/store executes, it checks the shadow byte corresponding to the redzone address. The check reveals the byte is poisoned, causing the runtime to intercept execution, print a detailed memory layout showing the overflow offset, and safely terminate the program.
  • Use-After-Free: If a pointer references memory that was passed to free(), the quarantined block's shadow bytes remain poisoned. Any subsequent read or write through the dangling pointer trips the instrumentation check, pinpointing the exact line where the stale reference was used, alongside stack traces for both the allocation and deallocation sites.

Through this combination of compiler-driven instruction rewriting, poisoned boundary padding, and direct shadow-memory indexing, memory sanitizers turn subtle, delayed heap corruption bugs into immediate, reproducible, and debuggable events.