How do you configure NVMe RAID on Amazon EC2 instances?

Configuring a software RAID array across multiple local NVMe instance store volumes on an Amazon EC2 Storage Optimized instance combines the throughput and input/output operations per second of individual drives into a single logical volume. You can accomplish this on Linux instances by identifying your unmounted NVMe block devices, assembling them into a RAID 0 or RAID 10 array using the mdadm utility, formatting the resulting virtual device with a filesystem such as XFS or ext4, and mounting it to a target directory. Because EC2 instance store volumes are ephemeral, data stored on local NVMe drives persists across standard instance reboots but is permanently lost if the instance stops, hibernates, or terminates.

Understanding EC2 Local NVMe Instance Store Volumes

Amazon EC2 Storage Optimized instance families—such as the I3, I3en, I4i, and Im4gn series—provide direct-attached Non-Volatile Memory Express (NVMe) solid-state drives (SSDs) physically installed in the host server. Unlike Amazon Elastic Block Store (EBS) volumes, which are network-attached, local NVMe instance store volumes deliver extremely low latency and high random I/O performance.

When combining multiple local NVMe drives into a Redundant Array of Independent Disks (RAID), two configurations are standard on EC2:

RAID 0 (Striping): Distributes data evenly across all drives in the array without parity or mirroring. If you have \(N\) drives each with capacity \(C\) and throughput \(T\), the total usable capacity is \(N \times C\) and the theoretical maximum throughput scales toward \(N \times T\). If any single drive fails or the instance stops, the entire array is lost. Because instance store volumes are already ephemeral, RAID 0 is the most common choice for high-performance scratch space, caches, or distributed databases that handle replication at the application layer.
RAID 10 (Mirrored Stripes): Requires an even number of drives (\(N \ge 4\)) and mirrors data across pairs of striped drives. The usable capacity is \(\frac{N \times C}{2}\). It provides fault tolerance against a single drive failure within each mirrored pair while maintaining high read and write performance.

Step 1: Identify the Local NVMe Drives

Before creating the array, connect to your EC2 instance via SSH and inspect the attached block devices to distinguish your root EBS volume from the ephemeral local NVMe drives.

Install the nvme-cli and mdadm packages if they are not already present on your Linux distribution:

# Amazon Linux 2023 / RHEL / CentOS
sudo dnf install -y mdadm nvme-cli

# Ubuntu / Debian
sudo apt-get update && sudo apt-get install -y mdadm nvme-cli

List the block devices and their mount points:

lsblk -o NAME,SIZE,TYPE,MOUNTPOINT,MODEL

You can also inspect the NVMe devices directly using nvme list:

sudo nvme list

On Nitro-based EC2 instances, both EBS volumes and instance store volumes appear as /dev/nvme*n1 devices. Look at the MODEL column in the output:

Amazon Elastic Block Store indicates an EBS volume (do not include your root volume in the instance store RAID array).
Amazon EC2 NVMe Instance Storage indicates an ephemeral local NVMe drive.

For the following steps, assume the instance has two unmounted local NVMe instance store volumes named /dev/nvme1n1 and /dev/nvme2n1. Verify that neither drive contains an active filesystem or mount point before proceeding.

Step 2: Create the RAID Array with mdadm

Use the mdadm utility to assemble the local NVMe drives into a virtual block device (such as /dev/md0).

To create a RAID 0 array across two NVMe drives (/dev/nvme1n1 and /dev/nvme2n1), run:

sudo mdadm --create --verbose /dev/md0 \
  --level=0 \
  --name=MY_RAID \
  --raid-devices=2 \
  /dev/nvme1n1 /dev/nvme2n1

If your instance has four local NVMe drives and you prefer RAID 10, specify --level=10 and --raid-devices=4:

sudo mdadm --create --verbose /dev/md0 \
  --level=10 \
  --name=MY_RAID \
  --raid-devices=4 \
  /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1 /dev/nvme4n1

Verify that the array is active and check its synchronization status by inspecting /proc/mdstat:

cat /proc/mdstat

You can also view detailed configuration parameters—including chunk size, UUID, and drive state—using:

sudo mdadm --detail /dev/md0

Step 3: Format the RAID Array with a Filesystem

Once /dev/md0 is initialized, create a filesystem on the array. XFS is generally recommended for high-throughput NVMe arrays on Linux.

Format the array with XFS and assign it a filesystem label:

sudo mkfs.xfs -L MY_RAID /dev/md0

Alternatively, if your workload requires an ext4 filesystem, run:

sudo mkfs.ext4 -L MY_RAID /dev/md0

Step 4: Persist the RAID Configuration and Mount the Volume

To ensure the operating system reassembles the RAID array automatically when the instance reboots, save the array layout to the mdadm configuration file and update the initial ramdisk.

Save the mdadm configuration:

# Amazon Linux / RHEL / CentOS
sudo mdadm --detail --scan | sudo tee -a /etc/mdadm.conf

# Ubuntu / Debian
sudo mdadm --detail --scan | sudo tee -a /etc/mdadm/mdadm.conf

Rebuild the initial ramdisk image so the boot process recognizes /dev/md0:

# Amazon Linux / RHEL / CentOS
sudo dracut -H -f /boot/initramfs-$(uname -r).img $(uname -r)

# Ubuntu / Debian
sudo update-initramfs -u

Next, create a mount point directory and mount the array:

sudo mkdir -p /mnt/raid
sudo mount /dev/md0 /mnt/raid

To mount the array automatically across reboots, retrieve the UUID of /dev/md0 using blkid:

sudo blkid /dev/md0

Open /etc/fstab in a text editor and add an entry using the volume's UUID or label. Always include the nofail mount option when configuring ephemeral EC2 instance store volumes in /etc/fstab. If the instance is ever stopped and started (which wipes instance store drives) or converted into an AMI launched on a different instance type, nofail allows the instance to boot normally even if the RAID array is missing:

UUID=<your-device-uuid> /mnt/raid xfs defaults,nofail,noatime 0 2

Test your /etc/fstab configuration without rebooting by unmounting and running mount -a:

sudo umount /mnt/raid
sudo mount -a
df -h /mnt/raid

Automating Setup Across Stop and Start Cycles

While data on local NVMe drives survives an OS-level reboot (sudo reboot), stopping an EC2 instance through the AWS Management Console or CLI releases the underlying host hardware. When the instance is started again, it migrates to a new host with blank NVMe drives, and the RAID superblocks will no longer exist.

If your architecture stops and starts instances regularly, you can configure a systemd service or an EC2 User Data script that checks whether /dev/md0 exists at boot, and if not, automatically runs the mdadm --create and mkfs.xfs commands before mounting /mnt/raid.