What Causes System Thrashing When Memory Is Exceeded?

System thrashing occurs when the combined working sets of active processes exceed available physical memory, forcing the operating system to spend most of its time swapping pages between RAM and secondary storage rather than executing instructions. This article explains the mechanics of working sets, how page-replacement feedback loops trigger thrashing, and how operating systems detect and mitigate the condition.

Understanding Working Sets and Physical Memory

In classical computing with virtual memory, each active process relies on a subset of its virtual pages to execute efficiently over a given time window. This subset is known as its working set, denoted as \(W_i\) for process \(i\). For all active processes to run without frequent disk stalls, the sum of their working sets must fit within the available physical memory frames \(M\):

\[\sum_{i=1}^{n} W_i \le M\]

When \(\sum_{i=1}^{n} W_i > M\), physical memory is overcommitted. Not every process can keep its active pages resident in RAM simultaneously, which guarantees that page faults will occur during normal execution.

The Page-Fault Feedback Loop

Thrashing is driven by a self-reinforcing feedback loop between CPU scheduling, page replacement, and disk I/O:

  1. Eviction of active pages: Because free frames are exhausted, the page-replacement algorithm (such as LRU or Clock) must evict resident pages to satisfy new allocations. When memory is overcommitted, evicted pages often belong to the active working set of another runnable process.
  2. Cascading page faults: When a process resumes execution and references a page that was just evicted, it triggers a major page fault. The CPU suspends that process and waits for the operating system to fetch the page from swap space on disk.
  3. Misleading CPU utilization: As more processes block on disk I/O, CPU utilization drops sharply. In systems with naive medium-term schedulers, low CPU utilization is interpreted as a sign that more work can be admitted, so the scheduler increases the degree of multiprogramming by starting or resuming additional processes.
  4. Worsening contention: Adding more processes increases \(\sum W_i\) further, shrinking the share of frames available to each process and driving the page-fault rate even higher.

Because disk or SSD access latency is orders of magnitude slower than RAM access, the system spends nearly all of its time transferring pages back and forth rather than making forward progress on user instructions.

Global vs. Local Page Replacement

The severity of thrashing depends partly on how the operating system allocates frames:

  • Global replacement allows any process to take a frame from any other process. While flexible, it enables one memory-hungry process to steal frames from others, causing the entire system to thrash together.
  • Local replacement restricts each process to its own allocated pool of frames. This isolates thrashing to the overcommitted process, preventing it from degrading unrelated workloads, though it can underutilize memory if pools are sized statically.

How Operating Systems Mitigate Thrashing

Modern operating systems break the thrashing cycle by controlling the degree of multiprogramming and managing working sets directly:

  • Working-set model: The OS tracks each process's recent page references and only schedules processes whose combined working sets fit within \(M\). If \(\sum W_i > M\), one or more processes are suspended (swapped out) so the remaining processes can run efficiently.
  • Page-fault frequency (PFF) control: The OS monitors the rate of page faults per process. If a process's fault rate exceeds an upper threshold, it is granted more frames; if no free frames exist, a process is suspended to free memory.
  • Out-of-memory (OOM) killer: In systems without sufficient swap space or when thrashing persists, the kernel may terminate a selected high-memory process to restore system responsiveness.