Why Distributed Caches Like Memcached Support UDP

This article explores why high-performance distributed caching systems such as Memcached implement User Datagram Protocol (UDP) support alongside traditional TCP. Distributed caches utilize UDP to minimize network latency, eliminate connection state overhead, and maximize throughput for read-heavy operations, trading transport-layer reliability for raw speed in environments where occasional packet loss can be safely tolerated.

1. Minimal Latency and Zero Connection Overhead

Transmission Control Protocol (TCP) requires a three-way handshake (SYN, SYN-ACK, ACK) to establish a connection before any data transfer begins, along with teardown handshakes when closing. In microservice architectures where thousands of short-lived client threads query the cache, this connection churn creates significant CPU and latency overhead.

UDP is a connectionless protocol. Clients can transmit a cache lookup query in a single packet immediately without prior negotiation, receiving the response in another single packet. This eliminates round-trip connection delays and results in the lowest possible lookup latency.

2. Lower Server-Side Resource Consumption

Maintaining hundreds of thousands of concurrent TCP connections consumes substantial operating system resources. Each active TCP connection requires memory for socket buffers, sequence tracking, window management, and congestion control states.

By using UDP, a caching server does not need to maintain state for connected clients. The cache simply processes incoming datagrams, extracts the key, retrieves the data from memory, and sends a reply datagram back to the source IP and port. This drastically reduces memory overhead and kernel-level context switching on the cache servers, allowing them to handle significantly higher request rates per node.

3. Alignment with Idempotent Cache Lookups

In typical distributed caching architectures, read requests (such as GET commands) are idempotent and non-critical:

• Safe Cache Misses: If a UDP packet carrying a read request or response is dropped due to network congestion, the application can either retry the request or treat the failure as a standard cache miss and fetch the authoritative data directly from the primary database.
• Stateless Operations: Because a dropped GET query does not alter system state, the strict delivery guarantees and automatic retransmissions provided by TCP are not strictly necessary at the transport layer.

4. Optimized for Small Key-Value Payloads

Distributed caches are primarily used to store small data fragments, such as session tokens, user profiles, or pre-rendered UI components. When the payload fits within a single network Maximum Transmission Unit (MTU)—typically around 1,500 bytes on standard Ethernet networks—UDP operates at peak efficiency by transmitting the entire request or response in a single datagram without triggering IP fragmentation.

Practical Trade-offs and Modern Usage

While UDP offers distinct performance advantages, it comes with limitations that require careful implementation:

• Application-Level Sequencing: Large values that exceed the MTU require multi-packet management, requiring the caching protocol to implement custom framing, request IDs, and sequence numbers.
• Lack of Flow Control: Unlike TCP, standard UDP lacks built-in congestion control, meaning network congestion can lead to dropped packets without automatic backoff.
• Security Considerations: Unprotected UDP services can be abused in distributed denial-of-service (DDoS) amplification attacks. Consequently, UDP-enabled caches must always be confined to secure, isolated internal networks or virtual private clouds (VPCs) rather than exposed directly to the public internet.