Linux Process Memory Allocation Explained

The Linux operating system manages memory allocation for user processes through a sophisticated combination of virtual memory abstraction, user-space allocators, and kernel-level paging mechanisms. When a program requests memory, the kernel does not immediately assign physical RAM; instead, it establishes virtual address mappings and delays physical allocation until the memory is actively accessed. This article explains how user processes request memory, how the kernel bridges virtual and physical memory via page tables, and how mechanisms like demand paging and overcommit maintain system stability.

Virtual Address Space Layout

Every user process in Linux operates within its own isolated Virtual Address Space (VAS), shielded from other processes. On a standard 64-bit architecture, this space is logically divided into distinct segments:

• Text (Code) Segment: Contains the executable machine code, mapped as read-only to prevent modification.
• Data and BSS Segments: Store initialized and uninitialized global and static variables.
• Heap: Grows dynamically upward toward higher memory addresses for runtime allocations.
• Memory Mapping Segment: Houses shared libraries, mapped files, and anonymous memory mappings created during execution.
• Stack: Grows downward toward lower memory addresses to manage local function variables, arguments, and return addresses.

Because each process operates in this virtual abstraction, memory addresses referenced by a program do not correspond directly to physical RAM locations.

User-Space Allocation: The Role of glibc

User applications rarely interact with the Linux kernel directly for every memory request. Instead, standard libraries like GNU C Library (glibc) provide memory management functions such as malloc(), calloc(), realloc(), and free().

Calling malloc() directs the user-space allocator to check its internal memory pools for an available chunk of the requested size. If suitable memory is found, it is returned without kernel intervention, reducing expensive context switches. If the existing pool is exhausted or the requested allocation is large, the allocator requests memory from the kernel using one of two primary system calls:

• brk() / sbrk(): Adjusts the program break point, moving the boundary of the heap segment upward to provide contiguous space for small-to-medium allocations.
• mmap(): Creates a new, independent virtual memory mapping in the memory mapping segment. Allocators typically use anonymous mmap() calls for allocations exceeding a specific threshold (commonly 128 KB by default in glibc).

Demand Paging and Page Faults

Linux employs a strategy called demand paging (or lazy allocation). When the kernel extends the heap with brk() or creates a mapping with mmap(), it merely reserves entries in the process's virtual memory structures (vm_area_struct). Physical memory (RAM) is not yet assigned.

1. Virtual Reservation: The process receives a valid virtual memory pointer, but no physical page frame is assigned.
2. Access Attempt: When the process attempts to read from or write to that virtual address, the CPU’s Memory Management Unit (MMU) checks the page table and discovers no physical translation exists.
3. Page Fault Trigger: The MMU generates a hardware interrupt known as a page fault.
4. Handling the Fault: The kernel's page fault handler verifies that the address is within a valid virtual memory area:
   • Minor Page Fault: If the address is valid, the kernel allocates a 4 KB physical memory frame, updates the process's page table to point to it, and clears the page.
   • Major Page Fault: If the memory resides on disk (such as a file mapping or swapped-out page), the kernel initiates disk I/O to load the data into RAM.
   • Segmentation Fault: If the address is invalid or violates permissions (e.g., writing to a read-only segment), the kernel sends a SIGSEGV signal, terminating the process.
5. Execution Resumes: The CPU restarts the instruction that caused the fault, now successfully reading or writing to physical RAM.

Translation via Page Tables and the MMU

To map virtual addresses to physical addresses, Linux uses multi-level page tables (typically 4-level or 5-level paging on modern 64-bit systems). The CPU hardware traverses these tables via the MMU to resolve addresses.

Because reading multi-level tables in RAM for every memory access introduces latency, the CPU caches recent translations in a hardware cache called the Translation Lookaside Buffer (TLB). When a TLB hit occurs, address translation completes in a single CPU cycle.

Memory Overcommit and the OOM Killer

By default, Linux permits memory overcommit, granting virtual memory requests that exceed total physical RAM and swap space combined. This design accounts for the fact that processes routinely reserve far more virtual memory than they actively write to.

If processes concurrently access their allocated memory and physical resources become completely exhausted, the kernel cannot fulfill pending page faults. To prevent total system freeze, the kernel invokes the Out-Of-Memory (OOM) Killer. The OOM Killer evaluates running processes based on memory footprint, runtime, and the process oom_score, selecting and terminating a candidate process with SIGKILL to reclaim physical memory for the rest of the system.