How Linux Manages Persistent Volumes with Rook

This article provides a technical overview of how the Linux operating system handles persistent volumes when orchestrated by Rook within a Kubernetes cluster. It explains the mechanics between the Linux kernel, the Container Storage Interface (CSI), and the Rook Ceph storage backend, illustrating how raw network storage is translated into mounted, production-grade Linux filesystems for containerized workloads.

The Role of Rook and Ceph in the Linux Environment

Rook is an open-source, cloud-native storage orchestrator designed for Kubernetes. Rather than functioning as a storage provider itself, Rook automates the deployment, configuration, and management of Ceph—a battle-tested, distributed storage system.

When deployed on a Linux-based Kubernetes cluster, Rook runs as an operator. It configures Ceph daemons (Monitors, Managers, and OSDs) directly on Linux nodes. These daemons leverage underlying Linux storage resources, such as raw partitions, raw NVMe/SATA drives, or logical volumes managed by Linux LVM (Logical Volume Manager), using the BlueStore engine for low-overhead write paths.

CSI Drivers: The Bridge Between Kubernetes and the Kernel

When a user requests a PersistentVolumeClaim (PVC), Rook and Ceph utilize specialized Container Storage Interface (CSI) drivers—primarily ceph.csi.ceph.com for block devices (RBD) and shared filesystems (CephFS). The Linux operating system interfaces with these persistent volumes through the following step-by-step workflow:

1. Volume Provisioning: The Rook operator provisions the underlying storage asset within the Ceph cluster according to the defined StorageClass parameters.
2. Node Attachment: The CSI node driver on the target Linux host maps the remote Ceph storage resource to the host.
3. Format and Mount: The host Linux kernel exposes the storage as a device, formats it with a standard filesystem if required, and mounts it into the container's mount namespace.

Handling Block Storage with RADOS Block Device (RBD)

For standard ReadWriteOnce persistent volumes, Rook relies on Ceph RBD. The Linux OS handles this integration primarily through the native Linux kernel RBD module (rbd.ko):

• Kernel Mapping: The Ceph CSI node plugin communicates with the Linux kernel via sysfs to map a Ceph image. The kernel assigns this mapped network volume a local block device handle, typically located under /dev/rbd* (e.g., /dev/rbd0).
• Device Formatting: If the volume is newly provisioned, the Linux host formats the block device using standard local utilities, applying filesystems such as ext4 or XFS.
• Mounting and Namespaces: The Linux kernel mounts the formatted block device to a global directory managed by Kubelet (typically inside /var/lib/kubelet/plugins/kubernetes.io/csi/). From there, the Linux VFS (Virtual File System) bind-mounts the path into the target pod’s container mount namespace.

Handling Shared Storage with CephFS

For multi-node access (ReadWriteMany workloads), Rook orchestrates CephFS. Linux handles CephFS persistent volumes using one of two methods:

• Kernel Driver (ceph.ko): High-performance deployments use the in-tree Linux CephFS kernel module. The Linux kernel establishes direct network connections with Ceph metadata servers (MDS) and OSDs, presenting the remote POSIX-compliant filesystem as a native mount point on the node.
• FUSE Driver (ceph-fuse): When the running Linux kernel lacks the appropriate module version or required security capabilities, the CSI driver falls back to running CephFS in user space using the Linux FUSE (Filesystem in Userspace) subsystem.

I/O Execution and Teardown

Once mounted, all container I/O operations travel through the standard Linux I/O stack:

• I/O Routing: System calls (read, write, fsync) issued by applications enter the Linux VFS layer. For RBD, operations translate into block I/O requests handled by the kernel's network socket layer, which transmits RADOS operations directly to Ceph OSD nodes.
• Page Cache and Buffering: Linux manages host-level page caching for these persistent volumes just as it would for local storage, optimizing read and write speeds based on node memory availability.
• Unmapping and Cleanup: When a pod terminates, Kubelet and the CSI node plugin signal the Linux OS to unmount the volume, flush unwritten data, release the namespace bindings, and unmap the /dev/rbd* device to preserve data integrity across the cluster.