Using numactl to Bind Processes to Linux CPU Nodes

The numactl utility in Linux controls Non-Uniform Memory Access (NUMA) policy for processes and shared memory. In modern multi-socket and multi-core server architectures, system memory is divided into nodes assigned to specific CPU sockets. This article details the function of numactl, explaining how it binds processes and memory allocations to specific CPU nodes to minimize latency, prevent cross-node bus traffic, and maximize application throughput.

Understanding NUMA Architecture

In symmetric multiprocessing (SMP) systems with a NUMA architecture, a processor accesses its own local memory faster than non-local (remote) memory attached to another processor. When the Linux kernel scheduler migrates a process across different physical sockets, the process may frequently access remote memory across interconnects like Intel Ultra Path Interconnect (UPI) or AMD Infinity Fabric. This introduces latency and reduces memory bandwidth.

Core Function of the numactl Utility

The primary role of numactl is to override the default Linux scheduler behavior and explicitly define CPU affinity and memory allocation policies. By executing an application through numactl, an administrator guarantees that the execution threads and their corresponding memory pages stay bound to optimal hardware locations.

numactl achieves this through several core operational functions:

• CPU Node Binding (--cpunodebind or -N): Restricts the execution of the target process to the CPU cores belonging to a specified NUMA node or list of nodes.
• Physical CPU Binding (--physcpubind or -C): Binds the execution directly to specific physical CPU core IDs, offering granular control over core assignment regardless of node topology.
• Memory Binding (--membind or -m): Mandates that memory allocations occur strictly on the designated NUMA nodes. If memory on the selected nodes is exhausted, allocation fails rather than spilling over to remote nodes.
• Local Memory Allocation (--localalloc or -l): Directs the process to always allocate memory on the node of the CPU executing the request.
• Interleaved Allocation (--interleave or -i): Distributes memory round-robin across specified nodes, useful when a single thread requires more bandwidth than a single memory controller can provide.

Practical Usage Examples

Before binding processes, system administrators inspect the NUMA topology using:

numactl --hardware

This displays the number of available nodes, CPU core mapping per node, memory capacity, and the relative distance (latency penalty) between nodes.

To run a high-performance application bound exclusively to the CPUs and memory of NUMA node 0:

numactl --cpunodebind=0 --membind=0 /usr/bin/target_application

To bind a process to specific physical cores (for example, cores 0 through 3) while ensuring memory is allocated locally on the node hosting those cores:

numactl --physcpubind=0-3 --localalloc /usr/bin/target_application

Performance Benefits and Use Cases

Using numactl is a standard optimization technique in enterprise environments:

• Database Management Systems: Databases like PostgreSQL, MySQL, and Redis experience significant latency reductions when their processes are pinned to the same NUMA node where their memory buffers reside.
• High-Performance Computing (HPC): Parallel computing tasks benefit from predictable cache behavior and isolated memory paths, preventing resource contention between workloads.
• Virtualization: Hypervisors and container engines use NUMA pinning to align virtual CPUs (vCPUs) with specific host memory blocks, eliminating virtualization overhead caused by cross-node memory fetch cycles.