Build BPF Toolchain on OpenWRT

I set a single goal: build an OpenWRT eBPF toolchain end to end—compile, flash, and verify on real hardware with no guesswork.

I explain the steps I use: enable CONFIG_BPF, CONFIG_BPF_SYSCALL, CONFIG_NET_CLS_BPF, and CONFIG_NET_ACT_BPF in the kernel via make menuconfig. Then I install build deps on my Linux host, clone the official repo, and run make -j$(nproc).

After I flash the produced image from bin/targets, I SSH to the router, run opkg update, install bpftool, and confirm support with bpftool prog list. This yields a firmware image with ebpf enabled and user-space tools ready to run a simple program on the network.

I keep risks explicit: back up config, verify device target, and preserve settings when flashing to avoid downtime. I also measure CPU and performance impact so the change improves the system in a clear way.

Table of Contents

Key Takeaways

I set the goal: build, flash, and verify a firmware image with ebpf support.
Enable the four kernel options and compile from the official repo.
Flash from bin/targets, then install bpftool and list programs to verify.
Back up configs and confirm device targets to reduce risk.
Measure CPU and performance impact to validate gains.

What this guide covers and who should use it

I walk you through building firmware, enabling kernel hooks, and loading a program on hardware.

This guide shows what you will get: a working build pipeline and a firmware image that can run verified eBPF programs on a router. You will see where to attach a program and how to confirm it runs on the network path.

Who should use this: Linux admins and engineers who manage routers, need packet visibility or policy offload, and can work at the command line. You should be comfortable with Git, shell, SSH, and basic router maintenance at an intermediate level.

Scope and tools: I cover build environment setup, kernel options for eBPF features, firmware compile, safe flashing, and a first program load with a real example. For loaders and inspection I reference libbpf, bcc, bpftrace, and bpftool.

Use cases: packet filtering, flow accounting, latency tracking, and custom network handling.
Safety: the kernel verifier enforces constraints and the JIT speeds execution—this keeps the system safe while you run useful programs.
What I won’t do: GUI-only workflows or abstract theory—you will build the software and run a program on hardware.

Prerequisites, hardware support, and backups

I start with a strict checklist. Confirm the router model and flash size. Match the image profile to the device before flashing.

Check storage and RAM. Building on the host needs tens of gigabytes. The router must have enough flash for the kernel and packages. It also needs RAM for map memory and verifier work.

Supported routers, storage, and CPU considerations

Verify target: model, exact flash layout, and supported image version. Only flash images that match the model.

Note CPU limits. Small cores hit performance bounds — XDP generic mode can cost cycles. Pick hooks with your cpu in mind.

Build host requirements and required packages

Use a modern Linux distro with stable Internet. Install packages: build-essential, git, libncurses5-dev, gawk, gettext, unzip, file, and zlib1g-dev.

Network access, SSH, and router configuration backup

Ensure reliable SSH and the correct management interface IP. Keep Ethernet ports and a serial console ready if available.

Export a LuCI backup and copy /etc/config via scp.
Record running version, kernel, and installed package list for post-update comparison.
Know the device failsafe/recovery method and confirm power stability — use a UPS.
Have SSH keys, root password, and serial tools ready.

Item	Check	Action
Router model	Exact hardware & flash size	Verify vendor model and supported image profile
Build host	Disk, packages, network	Install required package set and confirm Internet
Recovery plan	Failsafe, serial, UPS	Document recovery steps and secure power
System state	Version, kernel, config files	Export backups and save package list

Prepare the OpenWRT build environment from source

Get the build host ready: install the packages, clone the source code, and pick the right profile. Do this on a Debian/Ubuntu system for the commands below to work as written.

Install build dependencies (Ubuntu/Debian example)

Run a single command to pull the essential packages and save time:

sudo apt-get install build-essential git libncurses5-dev gawk gettext unzip file zlib1g-dev

Clone the repository and update feeds

Clone the official repo to track upstream kernel fixes and features:

git clone https://git.openwrt.org/openwrt/openwrt.git && cd openwrt
./scripts/feeds update -a && ./scripts/feeds install -a

Select target, profile, and packages

Run make menuconfig to choose target, profile, and kernel options. Pick a minimal package set—include bpftool and build helpers so the image remains small.

Save the .config to lock the build. Start the compile with make -j$(nproc). Watch the logs early for missing headers or feed failures to avoid wasting time. If you change target or core components, run make dirclean; otherwise keep incremental builds to speed the process.

OpenWRT eBPF toolchain components you need

I keep the component list tight: compiler, libs, user tool, and a few kernel flags. Each piece maps to a clear build or runtime need.

clang/LLVM and kernel headers

I install clang/LLVM and matching kernel headers so a program compiles to BPF bytecode that matches the linux kernel ABI. That prevents verifier failures and mismatched types.

libbpf and CO-RE

I use libbpf and CO-RE to build portable object files. CO-RE adapts to type differences across kernels so the same object attaches cleanly on devices with slight ABI changes.

bpftool

I include bpftool in the image. It is the single tool I use on-device to list programs and maps, check attach points, and dump verifier logs.

Enable CONFIG_BPF and CONFIG_BPF_SYSCALL to allow program and map syscalls.
Enable CONFIG_NET_CLS_BPF and CONFIG_NET_ACT_BPF to attach to TC and action pipelines for packet handling.
Rely on the verifier and JIT: verifier enforces safety; JIT gives near-native speed.
Keep the package list minimal—bpftool and libbpf userspace are enough for first deployments.

Component	Purpose	Practical step
clang/LLVM	Compile C to BPF bytecode	Install distro clang and kernel headers
libbpf + CO-RE	Load portable objects	Build with libbpf, use CO-RE macros
bpftool	Inspect and manage programs/maps	Include in image; run bpftool prog list

Finally, think about hooks: TC sees post-driver frames; select the hook that matches your features and packet visibility needs. Structure code with small functions and safe helpers for predictable implementation.

Enable eBPF in the kernel configuration

Before you compile, I lock down the kernel options that allow program loading and TC attachment.

Run make menuconfig and navigate to the kernel sections. Enable these symbols: CONFIG_BPF, CONFIG_BPF_SYSCALL, CONFIG_NET_CLS_BPF, and CONFIG_NET_ACT_BPF. These options let the system accept program loads and attach to the traffic control path.

Built-in vs modules — a practical choice

Built-ins: they load early and simplify attach at boot. Use them when you need predictable behavior and zero module ordering headaches.

Modules: they save flash space and let you keep the default image smaller. Expect extra load-order steps when userspace tools must appear before an attach.

Option	Pros	Cons
Built-in	Immediate attach, simpler boot	Bigger image size
Modules	Smaller base image	Must manage load order

Check your linux kernel version for moved symbols or new dependencies before you finalize. Document the configuration changes in a fragment. Save, exit, then run a config diff to confirm only intentional changes before you build.

Build, flash, and verify on your router

I move from build host to device and complete the compile-to-flash cycle with precise commands. I keep steps short and repeatable so you can reproduce results fast.

Compile firmware and packages efficiently

I run make -j$(nproc) to compile the kernel and package selection in parallel. This saves build time and uses all CPU cores.

Flash the image and preserve settings safely

Pick the correct file under bin/targets and choose the sysupgrade image for your device and version. Use the web UI or sysupgrade -v –keep to keep settings.

Never interrupt power during flash. Wait for the router to fully reboot before testing the system.

Install bpftool and confirm program support

SSH in, run opkg update then opkg install bpftool. Run bpftool prog list and bpftool feature probe kernel to verify kernel features and that you can load programs.

Check storage headroom and installed package integrity post-boot.
Record the running kernel and OpenWrt version strings for traceability.
Save build logs and sysupgrade output so you can roll back if needed.

Step	Quick action	Why
Compile	make -j$(nproc)	Faster, parallel build
Flash	Web UI or sysupgrade –keep	Preserve settings, safer
Verify	bpftool prog list	Confirm kernel supports BPF

Write, compile, and load a minimal eBPF program

I show a minimal C example that compiles to a BPF object and attaches to an interface. The goal: count packets safely and inspect results without breaking traffic.

Keep the source small: one function, one map, bounded loops only. This helps the verifier accept the program quickly and keeps runtime cost predictable.

Create the C source

Example source: a packet counter that returns XDP_PASS. Save as prog.c. Keep the function compact and use only supported helpers so verifier checks succeed fast.

Compile to BPF object

Compile with clang: clang -target bpf -O2 -c prog.c -o prog.o. This produces a BPF object the kernel will verify and run.

Load and attach

Load with bpftool:

bpftool prog load prog.o /sys/fs/bpf/my_prog type xdp
bpftool net attach xdp id <id> dev <ifname>

Alternative (TC ingress):

tc qdisc add dev <ifname> clsact
tc filter add dev <ifname> ingress bpf da obj prog.o sec prog

Attach mode selection

Pick a mode by driver support and needs:

XDP native — fastest when the NIC supports it.
XDP generic — works everywhere but slower.
TC ingress — stable across drivers and flexible for complex programs.

Use maps to exchange data

Expose counters via an array or hash map. Read map data with bpftool map dump or a tiny libbpf-based user program. Keep map structures simple to minimize verifier friction.

Step	Command	Why
Compile	clang -target bpf -O2 -c prog.c -o prog.o	Produce valid BPF bytecode for verifier
Load (XDP)	bpftool prog load prog.o /sys/fs/bpf/my_prog type xdp	Make the program available in the kernel
Attach	bpftool net attach xdp id <id> dev <ifname>	Bind the program to the interface
Inspect	bpftool prog show; bpftool map dump	Verify attach and read counters

Validate with tcpdump and bpftool prog show. Remove with bpftool net detach or by deleting the TC filter. Measure CPU and packet rates to confirm acceptable performance.

Kernel version features, performance tuning, and troubleshooting

I focus on practical kernel checks, performance tweaks, and fixes that stop regressions fast.

Match your program to the kernel version

Match helper availability and BTF types to the running kernel version. Mismatched types cause verifier rejects. I build with the same headers or use CO-RE to avoid failures.

XDP vs TC: driver impact and CPU trade-offs

XDP native is fastest when the NIC driver supports it. XDP generic works everywhere but can trigger SKB headroom copies—default 256 bytes—which raised CPU use in my MIPS tests.

Verifier, JIT, and program size

Keep loops bounded and instruction counts low so the verifier completes. Confirm the JIT is enabled—native code cuts runtime CPU costs versus interpreted execution.

Performance tips and common fixes

Reduce SKB headroom (32 vs 256 bytes) — lower copy overhead and improve throughput.
Use per-CPU maps for counters and LRU maps for bounded memory.
Measure with perf and iperf3; watch for copy-heavy stacks on small CPUs.

Area	Symptom	Fix
Load/Verifier	prog rejected	Rebuild with matching headers or CO-RE; shorten loops
XDP generic	High CPU, copy paths	Reduce headroom or use TC
Flash/modules	Unstable system after flash	Validate image, use failsafe recovery, avoid risky modules

Where to go next and how to extend your project

Move from experiments to a repeatable release: package a small libbpf-based loader so the program config and map setup run reliably at boot.

Organize your source and source code into a repo with CI that builds against matching linux kernel headers and the SDK. This catches regressions early and ties each version to a build artifact.

Automate tests in QEMU and on spare hardware to validate ebpf programs, counters, and path behavior. Export map data via a tiny CLI so users read state without shell access.

Ship an ipk package, plan upgrade windows, and publish a short post after milestones with version notes, performance data, and known issues. That keeps the project practical and easy to operate.

FAQ

What does "Build BPF Toolchain on OpenWRT" cover?

This guide shows how to build a complete BPF toolchain and kernel support from OpenWRT source. I walk through preparing a Linux build host, fetching and configuring the source, enabling kernel flags, compiling clang/LLVM and libbpf, and producing firmware and packages you can flash to a router. The focus is practical: compile, load, and verify packet-handling programs on real hardware.

Who should use this guide?

Engineers and sysadmins who run intermediate-to-advanced Linux networking stacks and need on-device observability or packet processing. You should be comfortable with cross-compilation, kernel configs, SSH, and flashing routers. This is not an introductory Linux networking tutorial — it’s hands-on and assumes you can follow build steps and debug toolchain issues.

Which routers and CPUs are supported?

Supported devices depend on OpenWRT targets and kernel support. MIPS, ARM, and x86_64 platforms are common. Check that the device has enough storage and RAM for additional packages, and verify CPU features like JIT suitability for performance. If you need offload or driver-specific XDP support, confirm the NIC and driver in upstream kernel releases.

What build host requirements and packages do I need?

Use a modern Debian/Ubuntu host with build-essential, git, subversion, gcc, clang, make, libncurses-dev, zlib1g-dev, python3, ccache, and rsync. You’ll also want clang/LLVM and binutils for compiling BPF bytecode. I list distro-specific commands in the prepare section so you can reproduce the environment exactly.

How should I back up the router before flashing?

Take a full UCI config export and any custom files under /etc. Save the current firmware and NVRAM settings if the device supports it. Use SSH to copy /etc and overlay contents to your build host. A tested fallback plan — serial recovery or TFTP — is essential in case flashing fails.

How do I clone the OpenWRT source and update feeds?

Use git clone of the official OpenWRT repo or a stable tag. Run ./scripts/feeds update -a and ./scripts/feeds install -a to pull package feeds. Keep the source tree and feeds in sync with the target release you intend to build. I recommend building from a release tag for reproducible results.

How do I select the correct target, profile, and packages?

Run make menuconfig. Choose the target system and subtarget that match your router. Pick a profile that matches the board. Then add required packages: clang/LLVM, libbpf, bpftool, and any runtime libraries. Configure kernel package options so they match the kernel image you will build.

Which components are required to compile and run BPF programs on the router?

You need clang/LLVM to compile C to BPF bytecode, kernel headers matched to the running kernel, libbpf for CO-RE and loaders, and bpftool for inspection and loading. Also enable kernel flags like CONFIG_BPF and networking hooks (NET_CLS_BPF, NET_ACT_BPF) so the kernel accepts and verifies programs.

What kernel configuration flags must I enable?

Enable core BPF support and syscall entry: BPF and BPF_SYSCALL. For networking, enable NET_CLS_BPF and NET_ACT_BPF for TC, and XDP and driver support if you plan XDP. Also enable CONFIG_BPF_JIT if you want JIT performance (check your CPU and stability).

Should I build BPF features as modules or built-in?

Modules give flexibility — you can load and unload without rebuilding the whole image. Built-in can reduce boot-time dependency issues. For rapid iteration, use modules; for minimal embedded images and early boot hooks, use built-in. I explain the trade-offs and how to toggle both choices in menuconfig.

How do I compile firmware and packages efficiently?

Use ccache and parallel make (-j) on a multi-core host. Build only required packages or use make package/packagename/compile to avoid full builds. Keep a shared dl/ and build_dir/ across builds to speed incremental work. I provide recommended make flags in the build section.

How do I flash the image while preserving settings?

Use the vendor or OpenWRT web/CLI upgrade tools that preserve settings, or export and reapply UCI config manually. If you must use sysupgrade, include the -v option to keep configuration files. Always test flashing on a spare device first and have serial/TFTP recovery ready.

How do I verify bpftool and kernel support after flashing?

Install bpftool package on the router and run bpftool prog show and bpftool map show. Check dmesg for verifier messages, and inspect /sys/kernel/debug/tracing if enabled. Running simple sample programs and checking they attach and run is the fastest verification step.

How do I write and compile a minimal packet-handling program?

Create a small C program that uses the BPF helper APIs and a section like SEC(“xdp”) or SEC(“tc”). Compile with clang targeting bpf -O2 -target bpf and link to libbpf headers to produce an object. I include a minimal example in the guide you can adapt for XDP or TC.

How do I load and attach the program?

Use bpftool to load and pin the object or use a libbpf-based loader. For XDP attach, choose between native and generic depending on driver support; for TC use tc filter add bpf. I show exact bpftool and tc commands and how to set attach modes.

When should I use XDP native vs XDP generic vs TC ingress?

XDP native gives the best performance but requires driver support and usually a newer kernel. XDP generic runs in soft XDP fallback and has higher CPU cost. TC ingress is more portable across drivers and integrates with existing qdisc pipelines. Choose based on driver compatibility and desired CPU vs latency trade-offs.

How do BPF maps let user space and kernel exchange data?

Create maps in the BPF program and access them via file descriptors in user space using libbpf or bpftool. Maps support counters, per-CPU values, hash tables, and arrays — pick the map type that matches your latency and memory needs. I cover map creation, pinning, and CRUD access patterns.

How do I match my program to the kernel version?

Use the kernel headers and CONFIG settings that match the target kernel. CO-RE with libbpf helps by allowing relocation against the target kernel’s types. Always test on the exact kernel version you will run on the router to avoid verifier rejects or missing helpers.

What are common verifier and JIT issues?

Typical problems: unbounded loops, excessive stack use, unsupported helper calls, and mismatched kernel helpers. JIT failures can occur on older CPUs or kernels. Inspect dmesg for verifier logs and simplify the program until the verifier accepts it. Use bounded loops and reduce stack variables to avoid rejects.

What performance tuning tips should I apply?

Pay attention to packet headroom, use per-CPU maps when possible, minimize map lookups, and prefer XDP for high packet rates. Offload to NIC hardware where supported. Measure with perf and tc counters; iterate on program size and helper usage to reduce CPU cost.

What are frequent build or flash failures and how do I troubleshoot?

Build failures often come from missing dependencies, mismatched headers, or incorrect feed versions. For flashing, common causes are wrong target/profile or insufficient flash space. Use build logs, kernel config diff, and serial console output to pinpoint the issue. I include diagnostic commands that reveal root causes quickly.

Where can I extend the project after getting a minimal program running?

Expand to observability (per-flow counters, latency histograms), enforcement (rate limiting, ACLs), or performance features (XDP redirects, hardware offload). Integrate with userspace collectors, Grafana, or Prometheus. The guide points to libbpf examples and kernel documentation for real-world patterns.

WilliamPatterson

I'm a lifelong tech enthusiast who is particularly passionate about Linux and open-source software. My journey into the world of technology began in college, when I first discovered the power of open-source communities. I graduated with a degree in Computer Science, having spent countless hours tinkering with Linux distributions and learning about the Linux Kernel.