Optimizing Redis for Game Matchmaking at Scale

This article explores how game backend engineers optimize Redis memory caches to handle millions of short-lived matchmaking tickets simultaneously. Handling this workload requires minimizing the per-ticket memory footprint, preventing memory fragmentation, and ensuring low-latency retrieval. Key techniques include adopting compact serialization formats, using memory-dense data structures, shortening key namespaces, implementing precise Time-To-Live (TTL) strategies, and configuring Redis-level memory management.

Efficient Data Structures and Encoding

Matchmaking systems often store metadata such as player MMR (Matchmaking Rating), latency profiles, team preferences, and ping metrics. How this data is structured within Redis directly affects memory consumption:

  • Hashes with Listpacks/Ziplists: Storing ticket attributes as Redis Hashes takes advantage of hash-max-listpack-entries (or hash-max-ziplist-entries in older versions). When a hash has few fields, Redis encodes it as a single contiguous memory block, eliminating much of the raw pointer overhead associated with standard hash tables.
  • Sorted Sets for Queues: When matchmaking involves searching across continuous parameters (such as MMR ranges), developers use Sorted Sets (ZSET). The player's MMR or queue entry timestamp serves as the score. This allows atomic range queries (ZRANGEBYSCORE) to pool compatible players while maintaining ordered ingestion.
  • Avoid Nested Schemas: Instead of maintaining multiple interrelated keys for a single ticket, flat schemas prevent redundant metadata allocations.

Compact Binary Serialization

JSON and XML introduce significant memory overhead due to repeated key strings, whitespace, and human-readable formatting. At a scale of millions of concurrent tickets, serialized text significantly bloats RAM usage.

  • MessagePack and Protocol Buffers: Replacing JSON with binary serialization formats like MessagePack, Protocol Buffers, or FlatBuffers reduces the payload size by 50% to 80%.
  • Bitfield Packing: Fixed-size attributes (e.g., role selections, region IDs, client versions) can be packed into raw bitfields or byte arrays, reducing fields that would otherwise require separate string or integer keys.

Key Namespace Minimization

Redis key names occupy valuable RAM. For millions of ephemeral tickets, long key names generate massive overhead relative to the actual payload:

  • Shortened Prefixes: Replace descriptive keys like matchmaking:ticket:us-east:user:987654321 with compact prefixes like m:t:987654321.
  • Implied Context: Regional or game-mode contexts should be handled via database partitioning, Redis Cluster key hashing tags (e.g., {na-east}:m:t:987654321), or separate Redis instances rather than deeply nested key prefixes.

Granular Expiration and Eviction Policies

Matchmaking tickets are inherently short-lived, with life spans ranging from a few seconds to a few minutes. Without strict expiration controls, stale data degrades cluster capacity:

  • Aggressive TTL Assignment: Every ticket must be created with an explicit expiration using commands like SET ... EX or pipeline-bundled EXPIRE. TTLs are typically set to match or slightly exceed the client's matchmaking timeout.
  • Proactive Eviction Settings: Configure the Redis eviction policy to volatile-ttl or volatile-lru. If the cache approaches its memory limit (maxmemory), Redis removes keys with an explicit TTL first, prioritizing the eviction of tickets closest to natural expiration.
  • Active Defragmentation: Ephemeral workloads with high write-and-delete churn cause memory fragmentation over time. Enabling Redis's active defragmentation (activedefrag yes) prompts the memory allocator (typically Jemalloc) to reclaim sparse memory pages without requiring an instance restart.

Pipeline and Scripted Operations

Network round-trips can degrade throughput and leave half-processed tickets in memory during matchmaking spikes:

  • Lua Scripting and Functions: Logic for searching, matching, and removing tickets can be executed atomically inside Redis via Lua scripts. This eliminates race conditions where two matchmakers claim the same ticket, preventing orphaned memory artifacts.
  • Pipelining: Batching read and write operations for incoming tickets reduces network I/O and CPU context switching on the Redis host, leaving more system resources available for memory handling.

Horizontal Scaling with Sharding

When ticket volumes exceed the memory or single-threaded limits of a standalone instance, developers partition workloads horizontally:

  • Redis Cluster and Hash Slots: Tickets can be distributed evenly across shards based on user IDs.
  • Functional Sharding: Separating matchmaking pools by game mode, platform, or geographic region into discrete Redis instances prevents a surge in one queue from starving memory resources across the entire game ecosystem.