What Happens When Memcached Memory Limit Is Reached?

When Memcached reaches its allocated memory limit and receives a request to store a new item, it frees space by initiating an automatic eviction process powered by a Least Recently Used (LRU) algorithm. Rather than rejecting new writes or expanding system memory, Memcached reclaims memory by identifying and deleting the oldest, least frequently accessed items within the specific memory class where the new item belongs.

Memory Allocation via the Slab Allocator

To understand how eviction occurs, it helps to examine how Memcached organizes memory. Memcached uses a slab allocation system to manage RAM efficiently and prevent memory fragmentation.

  • Slab Classes: Memory is partitioned into different slab classes, where each class handles items within a specific size range.
  • Slabs and Pages: Each slab class consists of 1 MB pages divided into fixed-size memory chunks tailored for items of similar sizes.
  • Slab Isolation: Items are stored strictly in the slab class that best fits their size. Consequently, memory management and eviction decisions occur independently within each specific slab class.

The LRU Eviction Mechanism

When a new key-value pair arrives and there are no empty chunks left in the corresponding slab class, Memcached enforces its LRU policy:

  1. Class-Specific Search: Memcached searches the LRU queue belonging strictly to the target slab class, rather than looking across the entire cache.
  2. Expired Item Removal: It checks the tail of the LRU queue for items that have already passed their expiration time (TTL) and immediately frees them.
  3. LRU Eviction: If no expired items exist, Memcached forcefully removes the item at the tail end of the queue—the item that has gone the longest without being accessed—to allocate space for the incoming data.

Flaws and Modern LRU Enhancements

In older versions of Memcached, standard LRU eviction suffered from a issue known as "crawler churn" or premature eviction, where rarely accessed large items could remain while frequently accessed items in another slab were evicted. Modern Memcached addresses this with an advanced Segmented LRU (2Q) system:

  • HOT LRU: New items enter here. Items in HOT cannot be bumped directly by fetch operations; they simply migrate to WARM as they age.
  • WARM LRU: Items that are accessed while in COLD move to WARM. This protects frequently accessed keys from sudden eviction.
  • COLD LRU: Items flow from HOT and WARM into COLD. Evictions only occur from the tail of the COLD queue.

Additionally, modern implementations include Slab Automove, a background thread that monitors slab class usage and reassigns unused memory pages from cold slab classes to high-demand classes, mitigating localized memory exhaustion.