Managing Linux Software RAID with mdadm

Linux manages software Redundant Array of Independent Disks (RAID) configurations through a combination of the kernel-level Multiple Device (MD) driver and the user-space utility known as mdadm. This system allows administrators to combine multiple physical storage drives into a single logical device to achieve data redundancy, increased performance, or both, without the need for an expensive hardware RAID controller. This article explains how the MD subsystem operates, how mdadm handles array creation and maintenance, and how the operating system detects and recovers from drive failures.

The Architecture: MD Driver and mdadm

Linux software RAID relies on a two-tier architecture:

1. The Kernel MD Driver: The low-level Multiple Device driver operates directly within the Linux kernel. It handles block-level I/O operations, calculates parity for levels like RAID 5 and 6, handles data striping across disks (RAID 0), and mirrors data blocks across drives (RAID 1).
2. The mdadm Tool: This is the administrative user-space CLI tool. It interacts with the MD driver to create, configure, monitor, assemble, and update RAID devices. While the kernel routes data, mdadm dictates how the array is organized and writes metadata—known as "superblocks"—to the participating storage partitions.

Array Creation and Metadata Management

When initializing an array, mdadm writes an on-disk superblock to each member drive. This superblock contains unique identifiers (UUIDs), the RAID level, array layout, and drive sequence details.

To create an array, an administrator runs:

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1

Once this command executes, the kernel creates a virtual block device (/dev/md0). File systems like ext4 or XFS are then built directly onto this virtual device, making the underlying striping or mirroring completely transparent to user applications.

Configuration and Boot Assembly

Because drive device letters (/dev/sdX) can change upon system reboot, Linux relies on mdadm metadata and configuration files to assemble arrays predictably:

• /etc/mdadm/mdadm.conf: This file stores the definitions and UUIDs of the arrays on the system. It helps the operating system scan and discover which physical devices belong to which virtual array.
• Initramfs Integration: If the root file system sits on a RAID array, the initial RAM filesystem (initramfs) includes mdadm binaries and scripts to assemble the array before mounting the root partition and handing off control to systemd.

Real-Time Monitoring and Status

Linux provides visibility into active arrays through both the /proc virtual file system and mdadm diagnostics.

• /proc/mdstat: Reading this file provides an instant snapshot of all active RAID devices, displaying their sync status, active disks, and rebuild percentages:

  cat /proc/mdstat

• Detailed Device Querying: Running mdadm --detail /dev/md0 outputs comprehensive health metrics, state flags (such as clean, active, or degraded), and a detailed list of participating partitions.
• Daemon Monitoring: The mdadm monitoring daemon (mdadm --monitor) runs as a background systemd service. It constantly watches kernel events and can send email alerts or execute custom scripts if an array degrades or a drive fails.

Failure Handling and Array Rebuilding

When a physical disk fails, the Linux MD driver marks the disk as faulty, transitions the array into a degraded state, and ceases writing to the damaged device while keeping the file system accessible.

The recovery workflow typically consists of three administrative actions:

1. Marking and Removing: The failed drive is flagged and detached:

   mdadm /dev/md0 --fail /dev/sdb1 --remove /dev/sdb1

2. Replacing the Hardware: The physical drive is swapped out, and an identical partition scheme is applied to the new drive.
3. Hot-Adding the Replacement: The new partition is added to the array:

   mdadm /dev/md0 --add /dev/sdb1

Upon adding the replacement device, the kernel MD driver automatically initiates a resynchronization process, calculating missing data via parity or reading from mirrored drives to restore full redundancy without system downtime.