Linux iostat: Monitor CPU and Disk IO Statistics

The iostat (Input/Output Statistics) command is a core utility within the Linux sysstat package used to monitor system input/output device loading by observing the time devices are active relative to their average transfer rates. This article explains how iostat accesses kernel-level data to generate real-time metrics, breaks down the core CPU and storage reports it produces, and demonstrates how system administrators use these statistics to diagnose hardware bottlenecks.

Where iostat Collects Its Data

iostat does not directly poll storage hardware or CPU registers. Instead, it reads counters maintained by the Linux kernel via the /proc pseudo-filesystem:

• /proc/stat: Provides system-wide CPU tick counters for time spent across user mode, system mode, idle time, and I/O wait states.
• /proc/diskstats: Tracks cumulative I/O operations, sectors read and written, time spent doing I/O, and current queue lengths for each block device.

Because these /proc files contain cumulative counters accrued since system boot, iostat computes statistics by taking snapshots. The first report generated reflects averages since system startup. If an update interval is specified (e.g., iostat 2), each subsequent report calculates the delta between intervals, dividing the changes by elapsed time to show real-time rates.

CPU Statistics Explained

By default, or when using the -c flag, iostat generates a CPU report that divides processor time into distinct percentages:

• %user: Percentage of CPU utilization while running user-level applications (un-niced).
• %nice: Percentage of CPU utilization running user processes with a modified priority.
• %system: Percentage of CPU time executing system-level (kernel) tasks.
• %iowait: Percentage of time the CPU was idle while the system had outstanding disk I/O requests. A high %iowait indicates that CPU capacity is constrained by slow storage operations.
• %steal: Percentage of time a virtual CPU spent involuntarily waiting for the hypervisor to allocate real physical CPU time.
• %idle: Percentage of time the CPU was idle with no outstanding disk I/O requests.

Disk I/O Statistics Explained

Using the -d flag displays disk utilization, and combining it with -x reveals extended metrics necessary for performance tuning:

• r/s and w/s: The number of read and write requests issued to the device per second.
• rMB/s and wMB/s: The volume of data read from or written to the device per second in megabytes.
• rrqm/s and wrqm/s: The number of read and write requests merged per second by the Linux I/O scheduler into single hardware requests.
• aqu-sz: The average queue length of requests issued to the device, including both requests in progress and requests waiting in the driver queue.
• await: The average time (in milliseconds) for I/O requests to be served. This includes the time spent waiting in the queue and the time spent servicing the request. Modern extensions break this down into r_await (read latency) and w_await (write latency).
• %util: The percentage of elapsed time during which the storage device had at least one active request. Values approaching 100% indicate that the device is saturated, though modern multi-queue devices (such as NVMe drives and RAID arrays) can often handle higher throughput even at high utilization levels.

Essential iostat Commands

• Basic interval sampling: iostat 2 5 (Collects metrics every 2 seconds, repeating 5 times).
• Extended device statistics: iostat -xz 1 (Displays extended I/O metrics every second, using -z to omit devices with zero activity during the interval).
• Human-readable megabytes: iostat -m (Displays read and write rates in megabytes per second rather than default blocks or kilobytes).