7-Zip NUMA Awareness on Multi-Socket Servers

Modern enterprise servers rely on Non-Uniform Memory Access (NUMA) architecture to balance memory bandwidth across multiple physical processors, making software-level NUMA awareness critical for performance. While 7-Zip is capable of utilizing dozens of CPU threads across multiple processor groups, it does not feature native, low-level NUMA memory allocation awareness. Consequently, running a single 7-Zip compression process across multiple sockets can lead to severe memory latency penalties and diminished scaling efficiency.

Multi-Socket Support vs. NUMA Awareness

There is an important distinction between distributing threads across sockets and being NUMA-aware:

  • Thread Distribution: Modern 64-bit releases of 7-Zip natively support Windows processor groups, allowing the software to detect and schedule compression threads across more than 64 logical cores and across multiple physical CPU sockets.
  • Memory Affinity (NUMA Awareness): True NUMA awareness requires an application to allocate memory explicitly on the local memory node associated with the core executing the workload (e.g., using VirtualAllocExNuma on Windows or libnuma on Linux). 7-Zip does not implement this explicit per-thread node binding.

The Performance Impact on Compression

The primary compression algorithm in 7-Zip, LZMA/LZMA2, is heavily dependent on fast memory access and large dictionary buffers. When 7-Zip runs across multiple NUMA nodes, several performance bottlenecks occur:

  1. Remote Memory Latency: Because 7-Zip relies on standard system memory allocators, worker threads on Socket 1 frequently end up reading and writing to dictionary buffers physically mapped to the memory controller of Socket 0. This introduces latency via inter-socket interconnects (such as Intel UPI or AMD Infinity Fabric).
  2. Interconnect Saturation: High-throughput dictionary matching across nodes causes heavy traffic over inter-socket buses, leading to bus congestion and core starvation.
  3. Diminishing Returns: While a single-socket system scales nearly linearly with additional threads, a multi-socket system running a unified 7-Zip task often plateaus or even regresses in total throughput once threads cross node boundaries.

Best Practices for Multi-Socket 7-Zip Workloads

To achieve optimal throughput on multi-socket hardware, users should manage affinity at the operating system level rather than relying on 7-Zip's automatic thread allocation:

  • Run Parallel Instances: Instead of running one archive job using all system threads, run independent 7-Zip processes concurrently, each processing a separate dataset.
  • Pin Processes to Nodes (Linux): Use the numactl utility to restrict both memory allocation and CPU execution to a single node: 
    numactl --cpunodebind=0 --membind=0 7z a archive1.7z /data1
numactl --cpunodebind=1 --membind=1 7z a archive2.7z /data2
  • Assign Processor Affinity (Windows): Use PowerShell, Task Manager, or utilities like start /affinity to bind specific 7-Zip instances to the cores corresponding to a single physical socket.