Linux Network Bonding and Link Aggregation

Network bonding in the Linux operating system is a kernel-level mechanism that aggregates multiple physical network interfaces into a single logical interface. This process, often referred to as link aggregation or NIC teaming, provides network redundancy to prevent single points of failure and increases total network throughput by balancing traffic across multiple connections.

The Architecture of the Bonding Driver

The Linux kernel implements link aggregation through the bonding kernel module. When loaded, this driver creates a virtual network interface (commonly named bond0, bond1, etc.) that acts as a master device. Physical network interface controllers (NICs) are then bound to this master device as slaves.

To the user space and higher-level networking protocols (such as TCP/IP), the bonded interface appears as a standard network adapter with its own IP address and MAC address. When an application transmits data:

1. The network stack sends the packet to the virtual bonding interface.
2. The bonding driver intercepts the packet and selects a slave interface based on the configured bonding mode and hash policy.
3. The selected slave's driver transmits the raw frame onto the physical wire.

For incoming traffic, slave interfaces receive packets and pass them to the bonding driver, which decapsulates and presents them uniformly to the operating system's network stack.

Bonding Modes

The bonding driver's behavior is dictated by its operational mode. Linux supports seven primary modes:

• Mode 0 (balance-rr): Transmits packets in sequential order, from the first available slave to the last. It provides both load balancing and fault tolerance, though it can cause packet reordering issues for TCP streams if packet arrival times vary across links.
• Mode 1 (active-backup): Only one slave interface remains active. Another slave becomes active only if the primary slave fails. The bond's MAC address is typically visible on only one port to prevent switch MAC table confusion.
• Mode 2 (balance-xor): Transmits based on a hash policy (such as [(source MAC XOR destination MAC) modulo slave count]). This ensures that traffic to particular destinations consistently uses the same slave, preserving packet ordering while providing fault tolerance and load balancing.
• Mode 3 (broadcast): Transmits all packets on all slave interfaces. This mode provides high fault tolerance at the expense of bandwidth efficiency.
• Mode 4 (802.3ad / LACP): Implements the IEEE 802.3ad Dynamic Link Aggregation standard. It creates aggregation groups sharing the same speed and duplex settings and utilizes the Link Aggregation Control Protocol (LACP) to negotiate link parameters with an upstream network switch. This mode requires switch support.
• Mode 5 (balance-tlb): Adaptive Transmit Load Balancing. Outgoing traffic is distributed according to the current load on each slave (calculated relative to interface speed). Incoming traffic is received by the current assigned slave. This mode does not require special switch support.
• Mode 6 (balance-alb): Adaptive Load Balancing. Combines transmit load balancing (Mode 5) with receive load balancing (RLB). Receive balancing is achieved at the ARP level, where the bonding driver intercepts ARP replies and overwrites the hardware address with the MAC address of one of the specific slaves.

Link Health Monitoring

The bonding driver continuously checks the integrity of its slave interfaces to manage failovers effectively. It does this via two mechanisms:

• MII Monitoring (miimon): Queries the interface's Media Independent Interface (MII) registers via the device driver to verify whether the physical carrier link is up. It is computationally light and fast, but it only detects local link failure (e.g., a severed cable directly attached to the host).
• ARP Monitoring: Periodically sends ARP queries to a designated upstream gateway or host. The driver determines link health based on whether ARP responses are received. This approach detects upstream route failures beyond the immediate physical switch port, though it generates periodic network overhead.

Through this modular combination of operational policies and health monitoring mechanisms, the Linux bonding driver delivers resilient, high-bandwidth networking suited for enterprise servers, hypervisors, and storage systems.