What Governs EC2 Aggregate EBS Bandwidth and IOPS Limits?

Understanding the performance boundaries of an Amazon EC2 instance is essential for designing storage-intensive workloads. Every EBS-optimized EC2 instance type enforces two distinct ceilings on attached Amazon Elastic Block Store volumes: maximum throughput (measured in megabytes or gigabits per second) and maximum input/output operations per second (IOPS). The actual storage performance achieved by an instance is always governed by whichever is lower between the aggregate capacity of the attached EBS volumes and the dedicated EBS network limits of the EC2 instance itself. This article explains the architectural constraints, bursting mechanisms, I/O block size relationships, and hardware ceilings that define these aggregate limits.

Instance-Level vs. Volume-Level Ceilings

Storage performance in AWS operates on a dual-constraint model. On one side, each individual EBS volume (such as gp3, io2, or st1) has its own provisioned IOPS and throughput limits. On the other side, the EC2 instance hosting those volumes has a dedicated network channel for EBS traffic with a hard ceiling on aggregate bandwidth and aggregate IOPS.

If you attach multiple high-performance EBS volumes to a single EC2 instance, their combined provisioned performance can easily exceed the instance's internal capabilities. When this occurs, the EC2 instance becomes the bottleneck. Mathematically, the effective maximum IOPS (\(I_{\text{effective}}\)) and effective maximum throughput (\(T_{\text{effective}}\)) can be expressed as:

\[I_{\text{effective}} = \min\left(I_{\text{instance}}, \sum_{i=1}^{n} I_{\text{volume}, i}\right)\]

\[T_{\text{effective}} = \min\left(T_{\text{instance}}, \sum_{i=1}^{n} T_{\text{volume}, i}\right)\]

In these equations, \(n\) represents the total number of EBS volumes attached to the instance. Even if ten gp3 volumes are configured to deliver \(16,000\text{ IOPS}\) each (an aggregate volume capability of \(160,000\text{ IOPS}\)), attaching them to an instance capped at \(40,000\text{ IOPS}\) will restrict total storage operations across all volumes to \(40,000\text{ IOPS}\).

Baseline vs. Burst EBS Performance

Modern EBS-optimized instances fall into two categories based on their size: those with fixed dedicated limits and those with burstable EBS limits.

Smaller and medium-sized instances within an instance family (for example, m6i.large through m6i.4xlarge) typically provide a baseline performance level and a higher burst ceiling. These instances can operate at their maximum burst bandwidth and IOPS for up to 30 minutes at a time, at least once every 24 hours, depending on a credit bucket system. Once the burst credits are exhausted, the instance's aggregate EBS performance drops back to its baseline bandwidth and baseline IOPS until credits are replenished during periods of below-baseline activity.

Larger instance sizes within the same family (such as m6i.8xlarge and above) do not rely on a burst bucket. Instead, their baseline performance is equal to their maximum performance, providing consistent, sustained aggregate EBS bandwidth and IOPS 24 hours a day.

The Relationship Between IOPS, Block Size, and Bandwidth

Aggregate EBS bandwidth and aggregate IOPS are not independent variables; they are linked by the I/O block size of the workload. Throughput is the product of the number of operations per second and the size of each operation:

\[\text{Throughput} = \text{IOPS} \times \text{I/O Size}\]

AWS specifies an instance's maximum EBS IOPS assuming a \(16\text{ KiB}\) I/O block size. However, AWS caps the size of a single EBS I/O operation at \(256\text{ KiB}\) for SSD-backed volumes (and up to \(1,024\text{ KiB}\) for HDD-backed volumes). Because of this relationship, an instance will hit one of the two limits first depending on the workload's average block size:

  1. Small Block Sizes (\(4\text{ KiB}\) to \(16\text{ KiB}\)): Transactional databases and metadata-heavy workloads generate small I/O operations. Under these conditions, the instance will reach its maximum aggregate IOPS limit well before saturating its available EBS bandwidth.
  2. Large Block Sizes (\(64\text{ KiB}\) to \(256\text{ KiB}\)): Sequential read/write workloads, such as data warehousing, log processing, or media streaming, issue large I/O blocks. Here, the instance will saturate its maximum aggregate bandwidth limit at a relatively low IOPS count.

For example, consider an instance with a maximum EBS bandwidth of \(1,250\text{ MB/s}\) (equivalent to \(10\text{ Gbps}\)) and a maximum IOPS limit of \(40,000\text{ IOPS}\):

  • At a \(16\text{ KiB}\) block size, reaching \(40,000\text{ IOPS}\) consumes \(40,000 \times 16\text{ KiB} = 640,000\text{ KiB/s} \approx 625\text{ MB/s}\), which is only half of the bandwidth limit. The IOPS ceiling governs performance.
  • At a \(64\text{ KiB}\) block size, \(40,000\text{ IOPS}\) would require \(2,500\text{ MB/s}\) of bandwidth. Because the instance is capped at \(1,250\text{ MB/s}\), the bandwidth ceiling restricts the workload to approximately \(20,000\text{ IOPS}\).

Underlying Hardware Architecture and the AWS Nitro System

The generation and underlying hypervisor architecture of an EC2 instance heavily dictate its storage performance ceilings:

  • AWS Nitro System: Newer generation instances built on the AWS Nitro System offload virtualization and storage I/O processing to dedicated Nitro Cards (custom ASICs). This hardware acceleration dramatically increases both maximum aggregate IOPS (reaching up to \(260,000\text{ IOPS}\) or more on high-end instances) and maximum EBS bandwidth (up to \(60\text{ Gbps}\) or \(80\text{ Gbps}\) on specialized storage and compute sizes).
  • Legacy Xen-based Instances: Older generation instances use software-based virtualization paths and exhibit significantly lower IOPS and bandwidth ceilings. On very old non-EBS-optimized instances, EBS traffic must even contend with standard network traffic over the same physical network interface.
  • Shared Network Bandwidth on Specific Instance Families: While most modern EBS-optimized instances have dedicated bandwidth for EBS separate from standard network traffic, certain ultra-high-network or specialized instance types share a unified physical network card ceiling between EC2 networking and EBS traffic. On those specific architectures, heavy network egress can reduce the available ceiling for EBS bandwidth.

Monitoring Instance-Level EBS Limits

To determine whether an EC2 instance is throttling your attached storage volumes, AWS provides dedicated instance-level CloudWatch metrics that should be monitored alongside volume-level metrics:

  • EBSByteBalance% and EBSIOBalance%: These metrics track the remaining burst credits for smaller EBS-optimized instances. If either metric drops to zero, the instance is restricted to its baseline EBS bandwidth or baseline IOPS.
  • EBSReadBytes and EBSWriteBytes: Summing these values over a period and dividing by the number of seconds reveals the current aggregate throughput, which can be compared directly against the instance's documented maximum bandwidth.
  • EBSReadOps and EBSWriteOps: Summing these values and dividing by the period duration yields the aggregate IOPS across all attached volumes, allowing you to verify if the workload is hitting the instance's IOPS ceiling.