How Do EC2 I-Series NVMe SSDs Achieve Low Latency and IOPS?

Local NVMe SSD storage on Amazon EC2 I-series instances delivers low latency and high random IOPS by attaching solid-state drives directly to the host server PCIe bus, bypassing network storage fabrics entirely. Paired with the AWS Nitro System and custom AWS Nitro SSD controllers, these instances eliminate virtualization overhead, network hops, and queuing delays. This architecture provides direct hardware access with up to 64,000 queues per device, enabling massive parallelism and consistent sub-millisecond response times for demanding database and analytics workloads.

Direct PCIe Attachment vs Network-Attached Storage

To understand the performance profile of Amazon EC2 I-series instances (such as I3, I3en, I4i, and I4g), it helps to contrast local Instance Store volumes with network-attached storage like Amazon Elastic Block Store (EBS). While EBS provides durable, persistent block storage over the network, every I/O request must traverse the host network interface, travel across the data center network fabric, and be processed by distributed storage servers before reaching the underlying media.

Local NVMe SSDs on I-series instances reside physically inside the same server enclosure as the compute CPU and memory. They communicate directly with the processor via the Peripheral Component Interconnect Express (PCIe) bus. Removing the network layer eliminates serialization delays, packet switching latency, and shared network bandwidth contention. Consequently, the total I/O service time (\(T_{\text{service}}\)) can be modeled simply as:

\[T_{\text{service}} = T_{\text{OS}} + T_{\text{PCIe}} + T_{\text{controller}} + T_{\text{NAND}}\]

By zeroing out network transit and remote storage processing times, I-series local storage brings average random read and write latencies down into the tens or hundreds of microseconds.

The Non-Volatile Memory Express (NVMe) Protocol

Legacy storage protocols such as SATA and SAS were designed in the era of spinning magnetic disks, where mechanical seek times dominated performance. Those older protocols rely on the Advanced Host Controller Interface (AHCI), which supports only a single command queue capable of holding 32 commands at once. While sufficient for a physical read/write head that can only be in one place at a time, a single queue creates a severe software bottleneck for modern solid-state drives made of parallel NAND flash dies.

The NVMe protocol was engineered specifically for non-volatile memory connected over PCIe. Instead of a single bottlenecked queue, the NVMe specification supports up to 64,000 independent submission and completion queues, with each queue supporting up to 64,000 pending commands.

In a multi-core EC2 I-series instance, the operating system assigns dedicated submission and completion queue pairs to individual vCPUs. This design yields several major performance advantages:

Lockless Parallelism: Each vCPU can submit I/O operations to its own queue without acquiring cross-core spinlocks or waiting on other threads.
MSI-X Interrupt Steering: Completion interrupts are routed back to the exact vCPU core that initiated the request via Message Signaled Interrupts-Extended (MSI-X), preserving CPU cache locality and avoiding inter-processor interrupts.
Streamlined Register Access: NVMe reduces the number of uncacheable PCIe register reads required per command compared to AHCI, cutting down CPU cycles spent per I/O operation.

Hardware Offloading with the AWS Nitro System

In traditional virtualized environments, a software hypervisor intercepts every guest I/O request, translates it, and forwards it to the physical storage device. At ratusan of thousands or millions of IOPS, this software emulation layer consumes massive amounts of host CPU cycles and introduces unpredictable latency jitter.

EC2 I-series instances are built on the AWS Nitro System, which offloads virtualization functions to dedicated hardware cards (Nitro Cards for Local NVMe Storage) and a lightweight hypervisor. Using Single Root I/O Virtualization (SR-IOV), the Nitro controller presents the local NVMe drives directly to the guest operating system as bare-metal PCIe devices.

When an application running on an I-series instance issues a read or write call, the guest OS NVMe driver writes directly to the hardware queues mapped into its memory space. The Nitro hardware handles line-rate AES-256 encryption and namespace isolation in dedicated silicon with zero impact on host CPU resources or I/O latency.

Custom AWS Nitro SSDs in Modern I-Series Instances

Starting with the I4i and I4g instance families, AWS introduced custom-designed AWS Nitro SSDs to address one of the hardest challenges in flash storage: tail latency (\(P_{99}\) and \(P_{99.9}\)).

Standard commercial SSDs rely on internal Flash Translation Layers (FTL) that perform background housekeeping tasks, most notably garbage collection and wear leveling. Because NAND flash pages cannot be overwritten in place—they must first be erased in large blocks—an SSD under heavy random write pressure must periodically read valid pages from fragmented blocks, write them to a new block, and erase the old block. When this garbage collection kicks in, incoming application I/O requests get stalled behind internal flash operations, causing sudden latency spikes.

AWS Nitro SSDs mitigate these spikes through tight integration between the drive controller firmware and the Nitro System:

Optimized Garbage Collection Scheduling: The custom firmware coordinates background flash management so it does not collide with peak foreground I/O bursts, reducing I/O latency variability by up to 75% compared to previous-generation I3 instances.
Dedicated Over-Provisioning: A reserved portion of raw NAND capacity is kept hidden from the user namespace, giving the controller ample free blocks to absorb sustained random writes without triggering immediate block erasures.
Real-Time Telemetry and Rapid Firmware Updates: Hardware-level integration allows AWS to monitor wear patterns and optimize flash media timing parameters across the fleet without disrupting running workloads.

How Queue Depth Drives High Random IOPS

To achieve the millions of random IOPS advertised on larger I-series instances, workloads must take advantage of the internal parallelism of the NAND flash architecture. A single NVMe SSD contains multiple flash channels, and each channel connects to multiple NAND dies capable of executing operations simultaneously.

The relationship between throughput (\(\text{IOPS}\)), average latency (\(L\)), and concurrency or queue depth (\(QD\)) is governed by Little's Law:

\[\text{IOPS} = \frac{QD}{L}\]

If a single 4 KB random read takes \(L = 100\ \mu\text{s}\) (\(0.0001\text{ seconds}\)), a single-threaded application issuing one request at a time (\(QD = 1\)) will achieve at most:

\[\text{IOPS} = \frac{1}{0.0001\text{ s}} = 10,000\text{ IOPS}\]

When an application increases its concurrency—by using asynchronous I/O engines such as Linux io_uring or libaio across multiple threads—it keeps dozens of requests in flight simultaneously (\(QD > 32\)). The NVMe controller distributes these concurrent requests across separate flash channels and dies in parallel, scaling total throughput to hundreds of thousands of IOPS per drive while keeping individual request latencies low.

Operating System and Application Tuning for Maximum Performance

Hardware capability alone is only part of the equation; the guest operating system and application stack must be configured to avoid reintroducing software bottlenecks:

Use Asynchronous Polling or io_uring: At very high IOPS rates, handling hardware interrupts for every completed 4 KB I/O consumes excessive CPU time in context switches. Modern Linux kernels support io_uring and NVMe polling (io_poll), allowing threads to check completion queues directly in user space without context switching.
Disable Heavy I/O Schedulers: Traditional Linux I/O schedulers designed to sort requests for spinning disks add unnecessary lock contention on NVMe devices. Setting the scheduler to none (or noop) allows requests to pass straight to the NVMe multi-queue (blk-mq) layer.
Align Filesystems and Block Sizes: Formatting local NVMe volumes with XFS or ext4 using the -E nodiscard (or discard mount options managed carefully) and aligning partition boundaries to 4 KB physical sector sizes prevents read-modify-write penalties inside the flash controller.
Stripe Across Multiple Drives with RAID 0: Larger I-series instances provide multiple physical NVMe devices. Combining them into a RAID 0 array using Linux mdadm or logical volume management aggregates the PCIe lanes and flash channels across all drives into a single high-throughput mount point.

Ephemeral Architecture Trade-offs

The extreme performance of EC2 I-series local NVMe storage comes with an architectural trade-off: ephemerality. Unlike EBS volumes, local Instance Store data is tied to the lifecycle of the physical host. Data persists across guest OS reboots, but it is permanently lost if the instance is stopped, hibernated, or terminated, or if the underlying hardware drive fails.

Because of this characteristic, I-series instances are purpose-built for workloads that either replicate data at the application layer or treat local storage as a high-speed scratchpad. Distributed NoSQL databases (such as Apache Cassandra, ScyllaDB, and DynamoDB-compatible engines), clustered search engines (Elasticsearch and OpenSearch), real-time analytics platforms (ClickHouse and Apache Spark), and high-frequency transactional caches rely on built-in quorum replication across multiple Availability Zones for durability, leaving the local NVMe SSDs free to deliver uncompromising raw I/O speed.