Why OS Boot Code Transitions CPU Modes for C

Early-stage operating system boot code must transition CPU modes because modern C compilers generate machine instructions that depend on architectural features—such as flat memory addressing, 32-bit or 64-bit registers, and standard runtime stacks—that do not exist in a processor's default power-on state. On architectures like x86, CPUs initialize in backward-compatible legacy states that strictly limit memory access and register width. Boot code written in assembly must initialize the CPU into a protected or long mode to provide the hardware environment that standard C code requires to execute.

The Hardware Reset State

When an x86 processor powers on or resets, it starts in 16-bit Real Mode to maintain backward compatibility with original 8086 processors. In Real Mode:

  • Register widths are restricted to 16 bits.
  • Memory addressing relies on a segmented scheme (segment * 16 + offset), limiting addressable physical RAM to a maximum of 1 megabyte (MB).
  • Hardware interrupts and memory protection mechanisms like paging and privilege rings are absent.

Standard C compilers, such as GCC or Clang, do not generate code for 16-bit segmented real-mode environments by default. Instead, they compile code assuming a flat, contiguous memory architecture where a pointer corresponds directly to a linear memory address.

The Requirements of the C Runtime Environment

Before executing compiled C code, the underlying hardware must fulfill several baseline assumptions made by the C abstract machine and target Application Binary Interfaces (ABIs):

  1. A Flat Memory Model: Modern C treats pointers as linear, uniform integers indexing across memory (e.g., 32-bit or 64-bit addresses). Segmented real mode requires explicitly manipulating segment registers (CS, DS, SS, ES) to access data beyond 64-kilobyte boundaries, which breaks standard C pointer arithmetic and data structures.
  2. Standard Stack Operations: Function calls in C rely on an active call stack to push stack frames, pass arguments, allocate local variables, and store return addresses. While a stack can exist in 16-bit mode, it is restricted to 16-bit alignment and boundaries. Modern C ABIs (such as the System V ABI) mandate 32-bit or 64-bit aligned stacks handled via dedicated stack pointer registers (ESP or RSP).
  3. Register Width and Datatypes: Standard C datatypes like int, long, and pointers are expected to match native processor widths (32 or 64 bits). In Real Mode, manipulating these types requires multi-instruction emulation, drastically reducing efficiency and breaking assumptions built into modern kernel toolchains.

The Role of Early Assembly Bootstrapping

To bridge the gap between hardware reset and high-level language execution, the bootloader or early kernel executes a sequence of low-level assembly tasks:

  • Enabling the A20 Line: Clears the legacy memory wrap-around at the 1 MB boundary to access extended memory.
  • Loading the Global Descriptor Table (GDT): Defines segment descriptors to create a basic flat memory model where code and data segments span the entire addressable space.
  • Switching the CPU Mode: Modifies control registers (such as setting the PE bit in CR0 for 32-bit Protected Mode, or enabling LME in EFER and activating paging for 64-bit Long Mode).
  • Setting Up the Stack: Explicitly sets the stack segment and loads a valid memory address into the stack pointer register (ESP or RSP).
  • Zeroing BSS: Initializes uninitialized global variables defined in C code to zero.

Once the processor operates in the appropriate 32-bit or 64-bit mode with a valid stack and flat addressing, the assembly code performs an indirect jump or call into the entry function of the C program (such as kernel_main), handing execution over to the compiled C environment.