How Linux Interacts with iSCSI Using Open-iSCSI

This article explains how the Linux operating system utilizes the Open-iSCSI initiator framework to discover, connect, and communicate with remote iSCSI storage targets over standard TCP/IP networks. It covers the dual-layer architecture spanning user-space tools and kernel modules, the process of target discovery and authentication, the encapsulation of SCSI commands into network packets, and how the OS presents remote Logical Unit Numbers (LUNs) as local block devices.

Architecture: User Space vs. Kernel Space

The Open-iSCSI subsystem splits responsibilities between user-space daemons and kernel-space drivers to optimize management and I/O performance.

• User Space: Driven by two primary components:
  • iscsid: The background daemon responsible for managing connections, processing login/logout negotiations, handling error recovery, and maintaining the Open-iSCSI configuration database (typically located in /var/lib/iscsi/).
  • iscsiadm: The command-line management interface used by administrators to discover targets, create session nodes, and initiate connections.
• Kernel Space: Composed of the core Linux SCSI mid-layer, the scsi_transport_iscsi transport class, and transport-specific drivers such as iscsi_tcp (software-based TCP/IP encapsulation) or hardware-assisted offload drivers (like bnx2i or qedi). The kernel handles raw I/O, serialization, and packet framing without entering user space.

The Discovery Phase

Before interacting with storage, the initiator must locate the available targets on the network.

1. Command Execution: An administrator runs iscsiadm -m discovery -t sendtargets -p <Target_IP>.
2. Target Interrogation: iscsid opens a temporary TCP socket (typically on port 3260) to the target's portal, issues a SendTargets query, and parses the returned list of available iSCSI Qualified Names (IQNs).
3. Database Population: The discovery results are parsed and stored as persistent target and node records in /var/lib/iscsi/nodes/.

Session Establishment and Authentication

Once targets are identified, the initiator transitions to session establishment:

1. TCP Handshake: iscsid establishes a permanent TCP/IP connection to the target portal.
2. iSCSI Login Phase: The initiator transmits an iSCSI Login Request Protocol Data Unit (PDU). During this stage, parameters like maximum data segment lengths, header/data digests, and immediate data allowances are negotiated.
3. Authentication: If configured, mutual or one-way Challenge Handshake Authentication Protocol (CHAP) occurs. The target issues a challenge, which iscsid signs using the credentials configured in /etc/iscsi/iscsid.conf or the node-specific record.
4. Handoff to Kernel: Once authentication and parameter negotiation succeed, iscsid transfers the established socket and session parameters to the scsi_transport_iscsi kernel module.

I/O Processing and Packet Encapsulation

Once the session is active, the Linux kernel processes storage operations through the standard storage stack:

1. Filesystem & Block Layer: An application issues an I/O request (such as a read or write). The filesystem translates this into block I/O requests (bio structures) and forwards them to the Linux block layer.
2. SCSI Mid-Layer: The block layer queues the request to the SCSI mid-layer, which converts the generic block I/O into standard SCSI Command Descriptor Blocks (CDBs).
3. iSCSI Transport Encapsulation: The iscsi_tcp module wraps the SCSI CDB into an iSCSI PDU. If data digests (CRC32C) are enabled, checksums are calculated and appended.
4. Network Transmission: The iSCSI PDU is passed down to the standard Linux networking stack (TCP/IP), encapsulated within TCP segments and IP packets, and transmitted out the physical Network Interface Card (NIC) to the storage target.
5. Response Handling: When the target replies with an iSCSI Response PDU, the kernel extracts the SCSI status and data payload, completes the SCSI command, and signals the originating application.

Device Representation in Linux

To the rest of the operating system, remote iSCSI LUNs behave identically to locally attached physical disks:

• SCSI Device Scanning: Upon successful session creation, the SCSI mid-layer issues a REPORT LUNS command to identify attached storage units.
• Device Nodes: The kernel maps each discovered LUN to a standard SCSI block device node (e.g., /dev/sda, /dev/sdb).
• udev Integration: The udev daemon automatically monitors these new block devices, generating persistent symlinks under /dev/disk/by-path/ and /dev/disk/by-id/ reflecting the iSCSI IQN and LUN identifiers.
• Mounting and Multipathing: Administrators can create filesystems directly on these nodes, use LVM, or bundle redundant paths into a single resilient device using device-mapper-multipath.

Session Maintenance and Failure Handling

iscsid continuously monitors the health of the connection using NOP-Out/NOP-In heartbeats:

• Keepalive Probing: At configured intervals, the initiator sends an iSCSI NOP-Out PDU. The target must respond with a NOP-In PDU.
• Handling Drops: If the network link drops or the target fails to respond within the replacement timeout window (node.session.timeo.replacement_timeout), the kernel pauses outstanding I/O queues and attempts session recovery.
• Failover vs. Error: If multipathing is enabled, the multipath daemon redirects the queued I/O to an alternate active path; otherwise, if the connection cannot be re-established before the timeout expires, the kernel fails the pending I/O operations back to the issuing process.