eBPF and bpftrace on Modern Debian Kernels

Modern Debian releases provide native support for extended Berkeley Packet Filter (eBPF) tracing tools like bpftrace by shipping kernels pre-configured with essential subsystem flags and embedded type information. This article explores how bpftrace operates within Debian's kernel architecture, detailing kernel configuration prerequisites, BTF (BPF Type Format) integration, the compilation and verification pipeline, and user-space instrumentation.

Kernel Configuration and Built-In Support

Debian kernels (from Debian 11 Bullseye onward, and prominently in Debian 12 Bookworm) are built with the necessary eBPF infrastructure enabled out of the box. Key kernel configuration flags include:

  • CONFIG_BPF=y and CONFIG_BPF_SYSCALL=y: Enables the core eBPF virtual machine and the bpf() system call for loading programs and creating maps.
  • CONFIG_BPF_JIT=y: Activates the Just-In-Time (JIT) compiler, translating eBPF bytecode into native machine instructions for minimal overhead.
  • CONFIG_BPF_EVENTS=y: Enables attachment of eBPF programs to kernel tracepoints, kprobes, and perf events.
  • CONFIG_DEBUG_INFO_BTF=y: Generates BPF Type Format metadata directly into the kernel binary, exposed via sysfs.

Historically, tracing tools required installing matching linux-headers or massive linux-image-*-dbg packages to parse kernel data structures. With CONFIG_DEBUG_INFO_BTF enabled in Debian's standard distribution kernels, structural debug information is exposed at /sys/kernel/btf/vmlinux. This allows bpftrace to inspect kernel types directly at runtime without external header dependencies.

The Execution Pipeline of bpftrace

When a user executes a bpftrace program on Debian, the execution flows through distinct user-space and kernel-space stages:

  1. Parsing and AST Generation: bpftrace accepts human-readable scripts or one-liners (using a syntax inspired by awk and C). The built-in lexer and parser construct an Abstract Syntax Tree (AST).
  2. Type Checking via BTF: The tool reads /sys/kernel/btf/vmlinux to validate structure definitions, function signatures, and field offsets accessed by the script.
  3. LLVM Intermediate Representation (IR): The AST is transformed into LLVM IR, which is then compiled into eBPF bytecode using LLVM libraries packaged in Debian.
  4. Program Loading via bpf(): The generated bytecode is passed to the kernel using the BPF_PROG_LOAD command of the bpf() system call.
  5. In-Kernel Verification: The kernel's eBPF verifier analyzes the bytecode to ensure it cannot crash the system. It checks for unbounded loops, out-of-bounds memory access, uninitialized register reads, and invalid pointer dereferences.
  6. JIT Compilation: Once verified, the JIT compiler translates the bytecode into native host architecture instructions (such as x86_64 or arm64).
  7. Attachment: The program attaches to the designated probe target (e.g., kprobes, kretprobes, tracepoints, uprobes, or perf hardware counters).

Data Collection and User-Space Communication

When an event triggers the probe, the kernel executes the JIT-compiled eBPF code. Instead of copying large amounts of raw trace data directly to user space, bpftrace uses eBPF maps—in-kernel key-value data structures such as hash tables, ring buffers, and histograms.

Aggregations, counts, and statistical calculations are computed directly inside kernel space. The user-space bpftrace process periodically reads these maps or consumes structured events from the BPF ring buffer, printing the formatted results upon script termination or at defined intervals. This minimizes context switches and ensures negligible overhead on production workloads.

Privileges and Access Control

By default, modern Debian kernels restrict eBPF usage to privileged accounts to reduce attack surfaces. Debian enforces this via the kernel.unprivileged_bpf_disabled sysctl parameter, which is set to 1 or 2 by default.

To run bpftrace, users must either execute commands as root (sudo) or possess specific Linux capabilities:

  • CAP_BPF: Allows loading eBPF programs and creating maps.
  • CAP_PERFMON: Permits performance monitoring and attaching to kprobes/tracepoints.
  • CAP_NET_ADMIN: Required if tracing network-related eBPF hooks.

Because of Debian's alignment with upstream kernel configurations and standard LLVM toolchains, tools like bpftrace function reliably as drop-in diagnostic solutions for tracing system calls, memory allocations, disk I/O latency, and scheduler activity.