Why C String Parsers Suffer From Off-by-One Bugs

Writing custom string parsers in C is notoriously susceptible to off-by-one boundary bugs due to the language’s lack of built-in bounds checking, manual memory management, zero-indexed arrays, and reliance on null-terminated byte sequences. This article examines the mechanics of these errors, highlighting how subtle mismatches between buffer allocations, loop conditions, and pointer arithmetic frequently introduce vulnerabilities into custom C parsers.

The Null Terminator Disconnect

In C, strings are not distinct data types with tracked lengths; they are contiguous arrays of characters terminated by a null byte ('\0'). This design creates a persistent mismatch between string length and memory footprint:

  • strlen(s) returns the number of visible characters, excluding '\0'.
  • sizeof(s) or the required allocation size must include the '\0'.

Developers frequently allocate memory using malloc(strlen(input)) instead of malloc(strlen(input) + 1). When the parser subsequently writes the final null terminator to conclude the parsed token, it writes one byte past the allocated boundary, corrupting adjacent heap metadata or data.

The Fencepost Problem in Loop Conditions

Parsing requires iterating through tokens, delimiters, and payloads. Translating iteration logic into loop bounds introduces the classic fencepost error, typically through the misuse of relational operators:

  • Using <= instead of <: Iterating over an array of size N from index 0 to N touches N + 1 elements.
  • Pre-increment vs. Post-increment: Using *ptr++ instead of *++ptr inside parser loops often advances the cursor before evaluating the termination condition, causing the pointer to read or write one byte beyond the valid region.

Because C allows raw pointer arithmetic, stepping past the end of an array does not trigger a runtime panic. The pointer silently accesses adjacent memory until undefined behavior, segmentation faults, or data corruption occur.

Lookahead and Lookbehind Traps

Parsing structured text (like JSON, HTTP headers, or CSV) often requires peek operations to determine the next state. A parser might inspect buffer[i + 1] to check for escape sequences (e.g., \") or two-character operators (e.g., ==).

If the parser checks buffer[i + 1] without first verifying that i is strictly less than length - 1, the read operation accesses unallocated memory or the byte immediately following the null terminator. Even if this read succeeds without a crash, the parser incorporates arbitrary memory values into its control flow.

Inconsistent Slicing and Tokenization

When extracting substrings, parsers compute offsets using calculations such as end_index - start_index. This leads to two common off-by-one pitfalls:

  1. Length vs. Distance: The distance between indices is end - start, but the number of elements in an inclusive range [start, end] is (end - start) + 1. Omitting the +1 truncates tokens; adding it unconditionally when the range is half-open [start, end) causes out-of-bounds reads.
  2. Buffer Overflows on Copy: When copying slices via memcpy or strncpy, developers must explicitly manage the destination's terminator. Failing to account for the extra space required for '\0' results in writing the terminator into the adjacent memory address.

Security Implications

In C, an off-by-one bug is rarely just an off-by-one bug. Writing one byte beyond an allocated boundary on the stack can overwrite the saved frame pointer (off-by-one frame pointer overwrite), potentially altering execution flow and leading to arbitrary code execution. On the heap, off-by-one errors can corrupt chunk headers, turning a small parsing oversight into a critical security vulnerability.