What Is the Purpose of EC2 Storage Optimized Instances?

The Storage Optimized instance family in Amazon EC2 is designed for workloads that require high, sequential read and write access to very large data sets on local storage. These instances are engineered to deliver tens of thousands of low-latency, random I/O operations per second to applications. By using locally attached Non-Volatile Memory Express solid-state drives or hard disk drives, they minimize storage access bottlenecks and maximize data throughput for intensive data processing tasks.

Core Architecture and Capabilities

Unlike general-purpose or compute-optimized instances that rely primarily on network-attached Amazon Elastic Block Store volumes for primary storage, Storage Optimized instances focus on instance store volumes physically attached to the host server. This architecture eliminates network hops between the compute layer and the storage media, resulting in significantly lower latency and higher input/output operations per second.

When evaluating storage performance, engineers often calculate throughput using the relationship between IOPS and the average I/O block size:

\[\text{Throughput} = \text{IOPS} \times \text{Block Size}\]

Storage Optimized instances excel at maximizing both variables in this equation, offering high IOPS for small, random transactions as well as high aggregate throughput for large, sequential data streams.

Key Instance Sub-Families

Amazon EC2 categorizes Storage Optimized instances into specific sub-families tailored to distinct storage access patterns and hardware configurations:

  • I3 and I3en Instances: Equipped with NVMe SSD-backed instance storage, these instances are built for low-latency, high random I/O performance, and high sequential read throughput. I3en instances offer lower cost per gigabyte of SSD instance storage and higher network bandwidth.
  • I4i and I4g Instances: Powered by AWS Nitro SSDs, these instances provide lower latency and reduced latency variability compared to previous generations. I4i runs on Intel Xeon Scalable processors, while I4g utilizes AWS Graviton2 processors for improved price performance.
  • Im4gn and Is4gen Instances: Also powered by AWS Graviton2 processors and AWS Nitro SSDs, Im4gn is balanced for compute and storage, whereas Is4gen is optimized for the lowest cost per terabyte of SSD storage and the highest density of SSD storage per vCPU.
  • D2 and D3/D3en Instances: These offer dense HDD storage designed for workloads that need high sequential read and write access to massive datasets, such as distributed file systems and data warehousing, at a low cost per terabyte.
  • H1 Instances: Built with high-capacity HDD storage (up to 16 TB) combined with high disk throughput and balanced compute power, making them suitable for MapReduce-based workloads and distributed file systems.

Primary Use Cases

Because of their direct-attached storage architecture and high throughput capabilities, Storage Optimized instances are suited for several enterprise workloads:

  1. NoSQL Databases: Databases such as Apache Cassandra, MongoDB, and Couchbase rely on fast local storage to handle high-velocity read and write operations with minimal latency.
  2. In-Memory and Transactional Databases: Relational databases, scale-out transactional databases, and Redis or Aerospike deployments that require fast local disk persistence and recovery.
  3. Data Warehousing and Big Data Analytics: Platforms like Apache Hadoop, Apache Spark, and Presto process massive volumes of data sequentially, benefiting from the high disk throughput of D-series and H1 instances.
  4. Search Engines and Log Analytics: Elasticsearch, OpenSearch, and Splunk clusters require rapid indexing and querying across terabytes of operational log data.
  5. Distributed File Systems: Systems such as Lustre, HDFS, and GlusterFS leverage dense local storage to serve large files across compute clusters efficiently.

Important Considerations for Ephemeral Storage

A critical operational aspect of the local instance store volumes provided by Storage Optimized instances is that the storage is ephemeral. Data written to local NVMe SSD or HDD instance stores persists only during the lifecycle of the instance. While the data survives guest operating system reboots, it is permanently lost if the instance is stopped, hibernated, or terminated, or if the underlying physical drive fails.

To maintain data durability when using Storage Optimized instances, architectures typically rely on application-level replication across multiple instances in different Availability Zones, or periodic snapshot backups to persistent storage services like Amazon S3 or Amazon EBS.