How the Linux du Command Calculates Directory Sizes

This article explains the internal mechanics of the Linux du (disk usage) command, detailing how it scans directories, interprets filesystem metadata, counts allocated disk blocks, and accounts for edge cases like hard links and sparse files to report directory space consumption.

Recursive Filesystem Traversal

The du command determines directory sizes by recursively traversing the target directory tree. When invoked on a path, it uses POSIX directory reading functions—such as opendir() and readdir()—to read the contents of the directory. For every item encountered within that directory, du issues a system call (typically fstatat() or lstat()) to retrieve file metadata directly from the filesystem's inode table.

If an entry is another directory, du recursively steps into it, repeats the inspection process for all nested files and subdirectories, and aggregates the results up the tree.

Block Allocation vs. Apparent File Size

Unlike tools that sum the apparent file length in bytes, du primarily measures the actual physical space reserved on the storage device.

When du retrieves the metadata structure (struct stat) for an inode, it reads the st_blocks field rather than st_size:

• st_size (Apparent Size): Represents the actual number of bytes written to the file.
• st_blocks (Disk Usage): Represents the number of 512-byte blocks allocated by the filesystem to store the file's data and metadata.

Storage devices allocate disk space in fixed-size chunks known as filesystem blocks (commonly 4096 bytes or 4 KiB). Even if a file contains only 10 bytes of data, the filesystem must allocate an entire 4 KiB block to it. Therefore, du counts the full 4 KiB block, reflecting true disk consumption.

This mechanism also naturally accounts for sparse files (files with unallocated "holes" containing zeroes). For a sparse file with an apparent size of 10 GB that only contains 100 MB of written data, du correctly reports the physical 100 MB usage based on st_blocks, ignoring the unallocated holes.

Accounting for Directory Inodes

In Unix-like systems, directories are specialized files containing tables of filenames mapped to inode numbers. Because of this, directories themselves occupy disk space on the storage device.

When calculating the total size of a directory, du includes:

1. The blocks allocated to the directory file itself (the space needed to store its entry list).
2. The blocks allocated to all regular files, symlinks, and special files inside it.
3. The blocks allocated to all subdirectories and their contents.

Handling Hard Links

To prevent inflating usage totals, du keeps track of visited inodes. When multiple filenames point to the same physical data on disk (a hard link), each entry shares the exact same inode number and device ID (st_ino and st_dev).

During execution, du maintains an internal hash table of every inode it has already counted. When it encounters a file with a link count (st_nlink) greater than one, it queries this table:

• If the inode has not been seen before, du records its size and adds the inode to the table.
• If the inode has already been processed in the current run, du skips its block count to ensure that the shared data is counted only once.

Behavior Modification with Flags

The internal calculation method changes when specific flags are provided to the command:

• --apparent-size: Directs du to read st_size instead of st_blocks. This sums the literal byte lengths of files, ignoring allocation block boundaries, sparse-file savings, and filesystem overhead.
• -b or --bytes: Automatically enables --apparent-size and displays values in raw bytes.
• -l or --count-links: Disables inode tracking, forcing du to count hard-linked files multiple times as if they were separate files.