Can EC2 Storage Optimized Instances Hit Millions of IOPS?

Amazon EC2 Storage Optimized instances can sustain millions of random read IOPS without network storage bottlenecks by utilizing locally attached NVMe SSD instance store volumes instead of network-attached storage like Amazon EBS. Because these drives are physically connected to the host server through the local PCIe bus, input and output operations bypass the network interface completely, eliminating network bandwidth ceilings and packet processing overhead. Modern instance families such as I4i, I4g, and I7ie leverage AWS Nitro SSDs and the AWS Nitro System to deliver up to 3.3 million or more random read IOPS at sub-millisecond latencies. This architecture makes them ideal for high-throughput, low-latency workloads such as NoSQL databases, real-time analytics, and distributed file systems that would otherwise saturate standard cloud network storage pipelines.

Local NVMe Instance Store vs. Amazon EBS Architecture

To understand how Storage Optimized instances bypass network bottlenecks, it is essential to distinguish between Amazon Elastic Block Store (EBS) and EC2 instance store volumes.

Amazon EBS provides persistent, block-level storage that is attached to an EC2 instance over the AWS network fabric. Even on EBS-optimized instances with dedicated network bandwidth for storage traffic, every read and write operation must traverse the virtualized network stack, leave the host server, and reach the distributed EBS storage fleet. While modern EBS volumes like io2 Block Express can achieve up to 256,000 IOPS per volume, scaling into the millions of IOPS over the network introduces two primary constraints:

1. Per-Instance EBS Bandwidth and IOPS Caps: Every EC2 instance has a hard ceiling on total EBS throughput and IOPS (for example, up to 260,000 to 400,000 EBS IOPS on the largest general-purpose or storage-optimized sizes).
2. Network Serialization Latency: Passing storage requests over a network fabric inherently adds serialization overhead, keeping round-trip latencies in the sub-millisecond to low-millisecond range (\(0.25\text{ ms}\) to \(1\text{ ms}\)).

In contrast, Storage Optimized EC2 instances feature NVMe instance store volumes. These are solid-state drives physically installed inside the same bare-metal chassis as the compute processors. Data transfers occur directly over the PCIe (Peripheral Component Interconnect Express) bus. Because the traffic never touches the Elastic Network Adapter (ENA) or the EBS storage network, network storage bottlenecks are completely removed from the data path.

The Role of the AWS Nitro System and Nitro SSDs

Achieving millions of random read IOPS requires more than just local physical drives; it requires eliminating hypervisor virtualization overhead. In legacy cloud architectures, the hypervisor intercepts storage I/O instructions from the guest operating system, translates them, and forwards them to the physical hardware. At millions of operations per second, this software translation layer becomes a severe CPU bottleneck.

AWS solves this through two custom hardware innovations:

• AWS Nitro System: The Nitro hypervisor offloads storage and networking virtualization to dedicated hardware cards (Nitro Cards). For local NVMe drives, the Nitro controller exposes the physical SSDs directly to the instance using SR-IOV (Single Root I/O Virtualization). The guest operating system interacts with the NVMe devices via bare-metal-equivalent PCIe pass-through, reducing CPU interrupt overhead and keeping I/O jitter near zero.
• AWS Nitro SSDs: Introduced in fourth-generation and newer storage instances (such as I4i, I4g, and Im4gn), Nitro SSDs are custom-built by AWS. Because AWS controls both the SSD firmware and the Nitro hardware controller, tasks like garbage collection, wear leveling, and encryption (via always-on XTS-AES-256 hardware encryption) are coordinated to prevent the latency spikes typically seen in commercial off-the-shelf SSDs under heavy random read/write loads.

IOPS Capabilities Across EC2 Storage Optimized Families

Different EC2 Storage Optimized instance families are engineered for specific ratios of compute, memory, and local NVMe performance. When benchmarking random read IOPS using \(4\text{ KiB}\) block sizes, top-tier instances easily cross the multi-million IOPS threshold:

For example, an i3.16xlarge or i3.metal instance distributes its \(15.2\text{ TB}\) of capacity across eight distinct \(1.9\text{ TB}\) NVMe SSDs. Similarly, an i4i.32xlarge or i4i.metal distributes \(30\text{ TB}\) across eight \(3.75\text{ TB}\) AWS Nitro SSDs. By striping reads across all eight PCIe-connected drives simultaneously, the instance aggregates the bandwidth and queue processing power of each independent NVMe controller.

Mathematical Relationship Between Latency, Queue Depth, and IOPS

Sustaining millions of random read IOPS on local NVMe storage is governed by Little's Law, which relates throughput (\(\text{IOPS}\)), average latency (\(L\)), and concurrency or Queue Depth (\(\text{QD}\)):

\[\text{IOPS} = \frac{\text{QD}}{L}\]

If a single NVMe operation over the local PCIe bus takes an average latency of \(L = 100\text{ }\mu\text{s}\) (\(0.0001\text{ seconds}\)), a single thread issuing synchronous reads (\(\text{QD} = 1\)) can only achieve:

\[\text{IOPS} = \frac{1}{0.0001\text{ s}} = 10,000\text{ IOPS}\]

To reach \(3,300,000\text{ IOPS}\) at a sustained average latency of \(150\text{ }\mu\text{s}\) (\(0.00015\text{ seconds}\)), the storage subsystem must maintain an aggregate in-flight queue depth across all drives of:

\[\text{QD} = \text{IOPS} \times L = 3,300,000 \times 0.00015 = 495\]

At a block size (\(B\)) of \(4\text{ KiB}\), the total PCIe bus throughput (\(T\)) required to sustain \(3.3\text{ million}\) IOPS is:

\[T = \text{IOPS} \times B = 3,300,000 \times 4\text{ KiB} \approx 13.2\text{ GiB/s}\]

Because a single PCIe Gen 4 x4 link provides nearly \(8\text{ GB/s}\) of raw unidirectional bandwidth—and large Storage Optimized instances attach up to eight NVMe drives across dedicated PCIe lanes—the PCIe interconnect easily accommodates \(13.2\text{ GiB/s}\) without saturation.

OS and Application Tuning to Reach Multi-Million IOPS

While the underlying EC2 hardware eliminates network storage bottlenecks, the guest operating system and application stack can quickly become the new bottleneck if not configured for extreme concurrency. Processing \(3\text{ million}\) IOPS means handling \(3\text{ million}\) completion events per second.

1. Software RAID 0 Striping

Large Storage Optimized instances present multiple discrete NVMe block devices (e.g., /dev/nvme1n1 through /dev/nvme8n1). A single drive cannot deliver \(3.3\text{ million}\) IOPS on its own. You must distribute the workload across all drives, either at the application layer (such as configuring multiple data directories in Apache Cassandra or ScyllaDB) or by uniting the drives into a RAID 0 array using Linux mdadm with an appropriate chunk size (typically \(64\text{ KiB}\) to \(256\text{ KiB}\)).

2. Using io_uring or libaio

Standard synchronous POSIX read calls (read() or pread()) incur context-switch penalties between user space and kernel space on every operation. To saturate multiple NVMe queues without exhausting CPU cores on system calls, applications must use asynchronous I/O interfaces:

• io_uring: The modern Linux asynchronous I/O framework that uses shared ring buffers between user space and kernel space, supporting polling modes (IORING_SETUP_IOPOLL) that eliminate hardware interrupt overhead entirely.
• libaio: The traditional Linux native asynchronous I/O library, using O_DIRECT to bypass the OS page cache and send requests directly to the NVMe submission queues.

3. Interrupt Affinity and NVMe Multi-Queue (blk-mq)

The NVMe specification supports up to 64,000 queues with 64,000 commands per queue. In Linux, the multi-queue block layer (blk-mq) maps individual NVMe submission and completion queues directly to specific vCPUs. Ensuring that the irqbalance daemon is properly distributing interrupts—or manually pinning application worker threads to the exact vCPUs coupled with the NVMe hardware queues—prevents cross-socket NUMA (Non-Uniform Memory Access) penalties.

4. Disabling File System Overhead

When testing or running custom database engines that manage their own block allocation, bypassing the file system (or using XFS with noatime and nodiratime mount options) prevents metadata locking from throttling high-concurrency random reads.

Architectural Trade-Offs: Ephemeral Storage Durability

The primary trade-off when using EC2 instance store volumes to achieve millions of IOPS is data persistence. Unlike Amazon EBS, whose volumes persist independently of the compute instance lifecycle, local NVMe instance store volumes are ephemeral.

Data on an instance store volume survives if the guest operating system reboots (via sudo reboot or the RebootInstances API). However, the data is permanently lost if:

• The instance is stopped or hibernated (StopInstances).
• The instance is terminated (TerminateInstances).
• The underlying physical host experiences a catastrophic hardware failure or drive failure.

Consequently, workloads running on Storage Optimized instances do not rely on AWS to replicate block data at the storage layer. Instead, durability is handled at the application layer using shared-nothing distributed architectures. Databases like Aerospike, ScyllaDB, Apache Cassandra, OpenSearch, and ClickHouse replicate data asynchronously or synchronously across multiple EC2 instances situated in different Availability Zones. Notice that while local random reads never touch the network, replicating writes between cluster nodes will utilize the instance's Elastic Network Adapter (ENA)—which on large I4i and I7ie instances supports up to \(75\text{ Gbps}\) or \(100\text{ Gbps}\) of network bandwidth to prevent inter-node replication bottlenecks.