How Nginx Handles High Concurrency on Linux

This article explores how the Nginx web server achieves high concurrency and low latency on the Linux operating system. By moving away from traditional thread-per-connection models, Nginx utilizes an asynchronous, event-driven architecture paired with Linux-native system calls to manage hundreds of thousands of simultaneous connections efficiently. Below is a breakdown of the architectural design, kernel interactions, and resource management strategies that make this performance possible.

The Master-Worker Architecture

Nginx operates using a master-worker process architecture rather than spawning a new process or thread for every incoming HTTP request.

• Master Process: Runs with elevated privileges to read configurations, bind to network ports, and manage worker processes. It handles lifecycle events like reloads, upgrades, and restarts without dropping client connections.
• Worker Processes: Run as unprivileged users and perform the actual work of accepting and processing client requests. Typically, the number of worker processes is configured to match the number of available CPU cores. By binding one worker per CPU core, Nginx virtually eliminates the operating system overhead of CPU thread context switching and ensures optimal CPU cache utilization.

Event-Driven, Asynchronous Processing

Traditional web servers often assign one thread or process to each client connection. Under heavy traffic, this approach consumes vast amounts of RAM and stalls CPU resources as threads block while waiting for network I/O or disk operations.

Nginx replaces this with an event-driven, non-blocking state machine. A single worker process does not wait for a network read or write to complete before moving to the next task. Instead, it continuously loops through an event queue, processing available data for thousands of connections sequentially. When an operation cannot proceed immediately—such as waiting for a client to send more data—the worker moves on to serve other active connections.

The Power of the Linux epoll Subsystem

At the heart of Nginx’s high-throughput capability on Linux is the epoll I/O multiplexing system call.

Older systems relied on select() or poll(), which operate in \(O(N)\) time complexity. With thousands of open connections, the operating system had to scan the entire list of file descriptors to find which ones were ready for reading or writing.

Linux epoll solves this by monitoring descriptors in \(O(1)\) time:

• Callback Registration: When an Nginx worker registers network sockets with epoll, the Linux kernel tracks them using a red-black tree.
• Ready List: Instead of scanning every descriptor, the kernel adds active descriptors to an event-ready list via hardware interrupts.
• Instant Notification: When Nginx queries epoll_wait(), the kernel returns only the file descriptors that have pending network events. This allows Nginx to handle 100,000+ idle and active connections with minimal CPU consumption.

Non-Blocking Disk I/O and sendfile

While network I/O is asynchronous via epoll, standard file access on Linux can still cause worker processes to block. Nginx overcomes this through specific Linux storage features:

• sendfile System Call: For serving static files, Nginx bypasses user space entirely. The sendfile() system call instructs the Linux kernel to transfer data directly from the disk page cache to the network socket buffer. This zero-copy mechanism avoids copying data back and forth between kernel space and user space.
• Asynchronous AIO and Thread Pools: For large files that could stall the event loop, Nginx can be configured to offload blocking file operations to a dedicated thread pool using Linux asynchronous I/O (aio).

Minimal Memory Overhead

Nginx is designed with strict memory management. A persistent connection requires only a tiny footprint—often less than a few kilobytes of RAM. Nginx uses pre-allocated memory pools for requests, headers, and buffers. This eliminates memory fragmentation and avoids the overhead of repeated malloc() and free() system calls during high-traffic spikes.