How do ext4 and XFS extents reduce metadata overhead?

Extent-based file systems like ext4 and XFS reduce metadata overhead by replacing individual block-by-block pointers with compact descriptors called extents. Instead of storing a separate address for every single data block in a large contiguous file, an extent records only the starting block, the logical offset, and the total length of the contiguous run. This approach dramatically shrinks the amount of metadata required on disk, accelerates file lookups, and minimizes the I/O operations needed to read or write large files.

Traditional Block Mapping vs. Extents

Older file systems, such as ext2 and ext3, rely on direct and indirect block mapping. An inode stores a small number of direct block pointers, followed by pointers to single, double, and triple indirect blocks. Each indirect block is a metadata block filled with pointers to other blocks.

For a large file of size \(S\) with a block size \(B\) and pointer size \(P\), the number of data blocks is:

\[N_{\text{blocks}} = \left\lceil \frac{S}{B} \right\rceil\]

With a standard \(4\text{ KiB}\) block size (\(B = 4096\text{ bytes}\)) and \(4\text{-byte}\) pointers (\(P = 4\text{ bytes}\)), a single indirect block holds \(1024\) pointers. A \(1\text{ GiB}\) file requires \(262,144\) data blocks, which translates to over \(256\) indirect metadata blocks just to map the file's layout. Every time the file is accessed or deleted, the file system must traverse and update these pointer trees.

How Extents Work in ext4 and XFS

An extent represents a contiguous range of physical blocks using a single metadata structure. Rather than listing blocks \(100, 101, 102, \dots, 5000\), the file system stores one entry: "start at logical block \(0\), map to physical block \(100\), for a length of \(4901\) blocks."

ext4 Extent Trees

In ext4, each extent is a \(12\text{-byte}\) structure (struct ext4_extent) that contains:

  • Logical block offset (\(32\text{ bits}\))
  • Length in blocks (\(16\text{ bits}\), allowing up to \(32,768\) contiguous blocks, or \(128\text{ MiB}\) with \(4\text{ KiB}\) blocks)
  • Physical starting block (\(48\text{ bits}\))

An ext4 inode has \(60\text{ bytes}\) of space for block mapping, which can hold up to \(4\) extents directly inside the inode. If a file is contiguous or has very few fragments, no external metadata blocks are needed at all. When a file grows beyond \(4\) extents, ext4 converts the inline list into a shallow B+ tree of extent blocks.

XFS B+ Tree Extents

XFS was designed around extents from the ground up. Each XFS extent record is \(128\text{ bits}\) (\(16\text{ bytes}\)) and encodes:

  • An unwritten extent flag (\(1\text{ bit}\))
  • Logical file offset (\(54\text{ bits}\))
  • Physical starting block (\(52\text{ bits}\))
  • Length in blocks (\(21\text{ bits}\), supporting up to \(2^{21} - 1\) blocks, or nearly \(8\text{ GiB}\) per extent with \(4\text{ KiB}\) blocks)

Like ext4, XFS stores small extent lists directly inside the inode's data fork. For larger or more fragmented files, XFS organizes extents into a B+ tree indexed by logical file offset, allowing logarithmic-time lookups even for files with millions of extents.

Key Performance and Space Benefits

  1. Reduced Metadata Footprint: A \(1\text{ GiB}\) contiguous file that requires hundreds of indirect blocks in ext3 can be described by just \(8\) extents in ext4 or a single extent in XFS, fitting entirely within the inode.
  2. Faster Sequential I/O and Deletion: Because fewer metadata blocks need to be read from disk to locate data, sequential reads and writes experience lower latency. Deleting a large file is also much faster, as the file system frees large contiguous ranges at once rather than walking thousands of individual block pointers.
  3. Improved Disk Cache Efficiency: Smaller metadata trees occupy less space in the kernel's page cache and buffer cache, leaving more memory available for actual file data.
  4. Synergy with Delayed Allocation: Both ext4 and XFS use delayed allocation, buffering writes in memory before assigning physical blocks. This allows the allocator to see the full size of a write operation and assign a single large contiguous extent, preventing fragmentation before it occurs.