How to Maximize Redis and Memcached Throughput on EC2 R-Series?

Maximizing throughput for distributed caching layers like Redis and Memcached on Amazon EC2 R-series instances requires aligning the software architecture with the underlying hardware resources. By tuning Linux kernel network parameters, selecting network-optimized R-series variants, configuring multi-threaded input and output processing, and applying efficient client-side patterns like pipelining, engineers can eliminate CPU and network bottlenecks to achieve millions of operations per second at sub-millisecond latencies.

Selecting the Right EC2 R-Series Variant

Amazon EC2 R-series instances are memory-optimized virtual machines designed for memory-intensive workloads such as in-memory caches, real-time big data analytics, and high-performance databases. They provide a fixed 1:8 vCPU-to-GiB memory ratio, ensuring ample RAM for caching large datasets while keeping compute costs predictable. However, because in-memory caching operations are computationally lightweight, distributed caches frequently saturate network bandwidth or packets-per-second (PPS) quotas long before exhausting memory or CPU capacity.

To maximize throughput, selecting the right instance generation and suffix is critical:

  • AWS Graviton Instances (R7g, R8g): Powered by custom Arm-based AWS Graviton processors, these instances provide significant price-performance improvements over x86 counterparts. Unlike Intel or AMD x86 instances where each vCPU is a simultaneous multithreading (SMT) hyperthread sharing a physical core, each vCPU on a Graviton instance is a dedicated physical core. This eliminates thread contention on L1/L2 caches and execution units, providing highly predictable latency and higher throughput per vCPU for both Redis and Memcached.
  • Network-Optimized Variants (R6in, R7i, R7g): Caching workloads are heavily bound by network I/O and interrupt processing. Instances with the n suffix (such as r6in) offer up to 200 Gbps of network bandwidth and dramatically higher packet-per-second allowances compared to baseline instances.
  • Elastic Network Adapter (ENA) Express: On supported R-series instances, enabling ENA Express utilizes the AWS Scalable Reliable Datagram (SRD) protocol to dynamically route packets across multiple network paths. While single-flow bandwidth limits still apply, SRD avoids congestion drops and tail-latency spikes during micro-bursts of cache traffic.

Overcoming Threading and CPU Bottlenecks

Redis and Memcached handle concurrency differently, and optimizing each engine requires aligning its threading model with the vCPU count of the R-series host.

Redis: I/O Threading and Multi-Instance Sharding

Historically, Redis executed all command processing and network I/O on a single main thread. While the single-threaded event loop avoids lock contention and context-switching overhead, a single core generally maxes out around 100,000 to 150,000 operations per second due to Linux socket read/write system call overhead.

In Redis 6.0 and later, you can offload network socket reads, command parsing, and packet writes to dedicated background threads while keeping data structure execution single-threaded. On an R-series instance with 8 or more vCPUs, configure the following parameters in redis.conf:

io-threads 4
io-threads-do-reads yes

A common rule of thumb is to set io-threads to roughly half or three-quarters of the available physical cores (generally between 4 and 8 threads; going beyond 8 threads rarely yields additional gains for a single process).

If your R-series instance has a large core count (for example, an r7g.4xlarge with 16 vCPUs or an r7g.8xlarge with 32 vCPUs), a single Redis process cannot saturate the machine even with I/O threads enabled. To utilize all vCPUs, deploy multiple Redis server processes per EC2 instance—each bound to a distinct port and pinned to specific CPU cores using taskset or the server_cpulist, bio_cpulist, and aof_rewrite_cpulist directives in redis.conf:

server_cpulist 0-3
bio_cpulist 4,5

Memcached: Scaling Worker Threads

Unlike Redis, Memcached is natively multi-threaded, using libevent to distribute incoming connections and command processing across worker threads. You configure the number of worker threads using the -t startup flag:

memcached -m 57344 -t 8 -c 16384 -o modern

While it is tempting to set -t equal to the total number of vCPUs on massive R-series instances, Memcached uses internal mutex locks around the slab allocator and the global hash table (segmented into lock buckets). When scaling beyond 8 to 16 threads under write-heavy or hot-key workloads, thread lock contention can degrade throughput. You can mitigate hash table lock contention by increasing the hash power multiplier (-o hashpower=24) and using the modern options profile, which enables segmented LRU queues and reduces global lock acquisition.

Linux Kernel and Network Stack Tuning

When processing hundreds of thousands of small TCP packets per second on an EC2 R-series instance, the default Linux kernel network stack becomes a major bottleneck. Applying the following sysctl and driver optimizations ensures the operating system can keep up with the ENA network card:

# Increase the maximum number of open file descriptors
fs.file-max = 2097152

# Expand the backlog of incoming connections waiting to be accepted
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535

# Increase the queue size for packets waiting for CPU processing
net.core.netdev_max_backlog = 65535

# Expand ephemeral port range and optimize TIME_WAIT socket handling
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_reuse = 1

# Increase TCP read and write buffer limits for high-bandwidth links
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216

# Enable TCP BBR congestion control for higher throughput and lower latency
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr

Beyond sysctl settings, you must optimize how the Elastic Network Adapter (ENA) distributes hardware interrupts across vCPUs:

  1. Receive Side Scaling (RSS) and IRQ Affinity: Ensure that the irqbalance daemon is either properly distributing ENA TX/RX queues across all vCPUs or manually pin ENA interrupt queues to dedicated cores that do not run the primary Redis event loop.
  2. Interrupt Moderation: Use ethtool -C to inspect and adjust adaptive RX/TX interrupt coalescing. Lowering rx-usecs reduces tail latency at the cost of higher CPU interrupt usage, whereas slightly increasing it under extreme PPS load batches packets per interrupt to maximize raw throughput.
  3. MTU and Jumbo Frames: Within an AWS VPC, configure the network interface to use an MTU of 9001 (Jumbo Frames) if caching larger payloads (above 1.5 KB). This reduces packet fragmentation and lowers the PPS burden on the ENA card.

Memory Management Optimization on High-RAM Instances

Because R-series instances offer anywhere from 16 GiB (r7g.medium) to over 1,500 GiB of RAM (r7i.48xlarge), operating system memory management settings have a direct impact on sustained throughput.

Disabling Transparent Huge Pages (THP)

Linux Transparent Huge Pages attempt to allocate 2 MB memory pages instead of standard 4 KB pages to reduce Translation Lookaside Buffer (TLB) misses. However, for in-memory caches with frequent small allocations and deallocations, THP causes memory fragmentation bloat and severe latency spikes during background compaction or when Redis forks a child process for snapshotting. Disable THP at boot time:

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

Tuning Virtual Memory and Swappiness

A caching layer must never swap to Amazon EBS storage, as even a single page fault over the network block device will stall an event loop thread for milliseconds. Set vm.swappiness to 0 (or 1 on newer kernels where 0 disables swap entirely) and set vm.overcommit_memory = 1 so background forks in Redis do not fail due to conservative kernel virtual memory checks:

vm.swappiness = 1
vm.overcommit_memory = 1

Preventing Fork Stalls and Defragmentation

To maximize pure caching throughput in Redis, disable disk persistence (save "" and appendonly no) on dedicated caching nodes. Without RDB snapshots or AOF rewrites, Redis never invokes the fork() system call, avoiding Copy-on-Write (CoW) page faults and allowing you to safely allocate up to 85%–90% of the R-series instance's memory via the maxmemory directive. Additionally, enable active memory defragmentation (activedefrag yes) with jemalloc to reclaim fragmented memory incrementally without pausing client traffic.

For Memcached, prevent page reassignment pauses by pre-allocating slab memory at startup (using the -L flag if configured with explicit Linux HugePages) and tuning the slab growth factor (-f, defaulting to 1.25) to match your application's object size distribution.

Client-Side and Protocol-Level Optimizations

Even the fastest EC2 R-series instance cannot achieve high throughput if clients interact with the cache inefficiently. By Little's Law, the maximum concurrency \(L\) required to sustain a target throughput \(\lambda\) given an average request round-trip time \(W\) is:

\[L = \lambda \cdot W\]

If a client executes commands sequentially over a single TCP connection with a network round-trip time \(W = 0.25\text{ ms}\) (\(0.00025\text{ s}\)), the maximum throughput for that connection is capped at:

\[\lambda = \frac{1}{0.00025\text{ s}} = 4,000\text{ ops/sec}\]

To multiply \(\lambda\) without opening thousands of connections, implement the following client patterns:

  • Pipelining and Batching: If a batch of \(N\) commands is pipelined in a single write syscall, the effective throughput per connection scales to:

\[\lambda_{\text{pipelined}} = \frac{N}{W_{\text{network}} + N \cdot T_{\text{exec}}}\]

where \(T_{\text{exec}}\) is the microsecond-level server execution time per command. Grouping 50 to 100 commands via Redis pipelines or multi-key operations (MGET, MSET in Redis; get with multiple keys in Memcached) reduces both network round trips and Linux kernel context switches between user space and kernel space (read()/write() syscalls), frequently boosting single-node throughput by an order of magnitude.

  • Connection Pooling and Multiplexing: Opening and closing TCP connections per request triggers TCP three-way handshakes (and TLS handshakes if encrypted), saturating CPU cores and exhausting ephemeral ports. Use persistent connection pools or asynchronous multiplexing clients (such as Lettuce for Java or redis-py async pools) to maintain a fixed number of warm connections.
  • Placement Groups: Deploy the EC2 instances running your application tier and your R-series caching instances within the same Availability Zone and inside a Cluster Placement Group. This ensures the instances are physically located on close hardware racks with non-blocking bisection bandwidth, minimizing \(W_{\text{network}}\) and allowing ENA interfaces to reach their peak packet-per-second ratings.