How Linux Uses Flannel for Kubernetes Overlay Networks

This article explores how the Linux operating system utilizes the Flannel network fabric to establish and manage Kubernetes overlay networks. It details the interaction between Flannel and native Linux networking subsystems, including network namespaces, virtual interfaces, routing tables, and the VXLAN kernel module, to enable seamless pod-to-pod communication across different cluster nodes.

The Role of Flannel in Kubernetes Networking

Kubernetes requires that every pod receives a unique, routable IP address and can communicate with any other pod in the cluster without Network Address Translation (NAT). Flannel satisfies this requirement by operating as a lightweight Container Network Interface (CNI) provider. It assigns a dedicated IP subnet (typically a /24) to each node from a larger cluster-wide address pool (typically a /16). While Flannel coordinates configuration via the Kubernetes API, it relies entirely on native Linux kernel primitives to execute packet encapsulation, routing, and delivery.

Pod Isolation and Host Attachment via Network Namespaces

The Linux operating system uses network namespaces (netns) to provide network isolation for individual Kubernetes pods. Each pod runs inside its own namespace with an isolated network stack, containing its own routing table, firewall rules, and loopback device.

To connect an isolated pod namespace to the host network:

  1. Virtual Ethernet Pairs (veth): The Linux kernel creates a pair of connected virtual Ethernet interfaces. One end of the pair stays inside the pod's network namespace (commonly named eth0), while the peer device remains in the host's root namespace.
  2. Local Bridging: On the host, the root-end veth interface is attached to a software bridge (such as cbr0 or managed directly by CNI bridge plugins). This allows packets leaving the pod namespace to enter the host's routing pipeline.

Encapsulation with the Linux Kernel VXLAN Module

Flannel's default and most common backend is VXLAN (Virtual Extensible LAN). Instead of implementing packet processing in user space, Flannel configures the native Linux VXLAN kernel driver to handle encapsulation at line speed.

When the Flannel daemon (flanneld) initializes on a node:

  1. It creates a virtual network interface in the host's root namespace, typically named flannel.1.
  2. This interface acts as a VXLAN Tunnel Endpoint (VTEP), which possesses its own MAC address and IP address corresponding to the node's allocated pod subnet.
  3. The Linux kernel listens on UDP port 8472 for incoming encapsulated overlay packets.

Routing Tables and Neighbor Discovery

When a pod sends traffic to a destination pod located on a different node, the packet follows a deterministic path managed by Linux kernel tables:

  1. Routing via flannel.1: The packet exits the pod via the veth pair onto the host. The host's Linux routing table has an entry managed by Flannel directing all cluster CIDR traffic to the flannel.1 device.
  2. Forwarding Database (FDB) Lookup: To know which physical host to target, the Linux kernel checks its internal Forwarding Database (FDB). Flannel populates the FDB with mappings of remote VTEP MAC addresses to the actual physical (underlay) IP addresses of the remote nodes.
  3. ARP Resolution: The host kernel checks its neighbor table (ARP cache), also pre-populated by Flannel, to resolve the remote pod's subnet gateway to the MAC address of the remote node's flannel.1 device.

End-to-End Packet Traversal

Once routing and neighbor information are evaluated, the complete transfer occurs as follows:

  1. Encapsulation: The Linux VXLAN driver takes the original Layer 2/3 frame (containing the source and destination pod IPs) and wraps it inside an outer UDP packet. The outer source IP is the local host's physical IP, and the outer destination IP is the remote host's physical IP.
  2. Transmission: The packet traverses the physical network fabric (the underlay) as standard UDP traffic.
  3. Decapsulation: Upon arrival at the destination node, the physical interface passes the UDP port 8472 packet to the Linux VXLAN module. The kernel strips away the outer UDP/IP headers to reveal the inner original packet.
  4. Local Delivery: The kernel reads the inner destination IP address and uses local routing rules to direct the packet across the local bridge and veth pair into the destination pod's network namespace.

Direct Routing Alternative: The host-gw Backend

When all Kubernetes nodes share direct Layer 2 connectivity on the underlying network, Flannel can utilize its host-gw (host gateway) backend instead of VXLAN. In this mode, Flannel bypasses encapsulation entirely. It programmatically adds direct routes to the Linux host routing table, designating the remote node's physical IP as the next-hop gateway for that node's assigned pod subnet. This approach reduces CPU overhead and eliminates MTU penalties associated with VXLAN encapsulation by relying purely on native Linux IP forwarding.