Capture Packets in OpenWRT with eBPF

I build and flash custom firmware so I can run a practical OpenWRT eBPF packet capture on small routers without guessing. I focus on concrete steps: kernel config flags, compiling from source, and verifying support with bpftool.

I explain three workable capture paths: quick XDP loads via iproute2 or xdp-tools, debugfs prints using bpf_trace_printk, and a user-space loader that polls perf stats. Each method trades CPU, RAM, and flash for observability.

I call out common pitfalls: endianness, header paths in staging_dir, and architecture mismatches. You’ll also see how HTTP tracing hooks accept4/read/write/close and pushes truncated payloads through perf buffers to limit overhead.

Table of Contents

Key Takeaways

I lay out reproducible steps: enable CONFIG_BPF options, build, flash, and install bpftool to verify.
Choose XDP, debugfs, or a user-space loader based on your router’s CPU and memory limits.
Watch for toolchain and header path issues when compiling BPF code for the kernel.
HTTP tracing uses syscall hooks and perf buffers with payload truncation to control traffic data size.
Use interface stats and bpftool prog list to confirm the system is processing events.

What you need before starting on OpenWRT

Start by pinning the target version and toolchain so feeds and headers resolve. Match target/arch to your router SoC. That avoids mismatched headers and failed builds.

I confirm device resources up front: free flash on the build host (hundreds of MB) and enough RAM on the router to load maps. Low-memory devices limit user space loaders and available space for images.

Supported version, toolchain, and router resources

Pick a supported firmware version that aligns with your hardware. Back up the router config and download the current image before flashing.

Build dependencies and SDK setup

Install build tools in one command on a Linux host: build-essential, git, libncurses5-dev, gawk, gettext, unzip, file, zlib1g-dev. Clone the source with this command:

git clone https://git.openwrt.org/openwrt/openwrt.git

Run make menuconfig and keep the source and SDK in the same workspace. Export STAGING_DIR and point includes to the SDK bpf-headers to build XDP objects without missing headers.

I choose libbpf for production and BCC for rapid prototyping based on device CPU and flash.
I validate the method with a dry run: make defconfig then make -j$(nproc) V=s.
I record command outputs and data paths so the image, kmods, and source code versions match.

Item	Recommended	Why it matters	Action
Version	Match target/arch	Feeds and headers resolve	Pin branch and tag
Build tools	build-essential, git, zlib1g-dev	Compile reliability	Install once on host
SDK	Same workspace as source	Avoid include path churn	Export STAGING_DIR
Loader library	libbpf (prod) / BCC (dev)	Performance vs prototyping	Choose per device limits

Enable kernel support and verify eBPF on your system

The first step is to enable BPF support in the linux kernel and lock config values. I set exact options in make menuconfig so the build and runtime match.

Kernel options to enable

Turn on these options:

CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
CONFIG_NET_CLS_BPF=m
CONFIG_NET_ACT_BPF=m

Without them, user loaders cannot attach programs and traffic hooks fail.

Build, flash, and confirm

Build with a parallel job run: make -j$(nproc). Firmware appears under bin/targets.

Flash the matching file via LuCI or sysupgrade, then reboot. On the device run:

opkg update
opkg install bpftool
bpftool prog list — should show loaded programs or accept a no-op load test.

Common build pitfalls

Problem	Symptom	Fix
Wrong image architecture	Boot failure or modules reject	Pick correct bin/targets file for device
Missing staging_dir headers	Out-of-tree code fails to compile	Export STAGING_DIR and use SDK bpf headers
Endianness mismatch	Wrong counters or map values	Serialize values for target endianness
Loader linked to wrong C library	Attach calls fail with vague errors	Cross-compile against matching libc and kernel headers

I keep kernel and version alignment tight. I log build output to a file so failures are easy to trace. Verify tc and XDP attach paths independently to isolate issues quickly.

OpenWRT eBPF packet capture: practical methods that work

I pick the simplest attach path first and escalate only when I need more live metrics.

I start with the quick way: ip link set dev eth0 xdp obj prog.o sec xdp_pass. It loads fast and shows immediate results.

When I need more control I use xdp-tools: xdp-loader load -m drv -d eth0 prog.o and xdp-loader status. That tool gives attach state and basic stats.

Debugfs and post-processing

If user space is tight I add bpf_trace_printk in the code and read /sys/kernel/debug/tracing/trace. It works on low-memory devices but needs careful rate control and offline parsing.

User-space loader and live stats

I write a light loader that calls attach_xdp_program and runs stats_poll. It reads rx_packets and rx_bytes, computes pps, and pins maps for reuse. Handle endianness and map sizes when cross-compiling.

Quick: iproute2 — minimal moving parts.
Managed: xdp-tools — attach and status APIs.
Fallback: debugfs — logs for tiny devices.
Reliable: user-space loader — live counters and map control.

Way	Command	Pros	Cons
Quick	ip link set … xdp obj	Fast, simple	No user stats
Managed	xdp-loader load / status	Status, APIs	Requires tool install
Debugfs	bpf_trace_printk → trace	Low memory	Verbose, post-process
User loader	attach_xdp_program; stats_poll	Live metrics, map control	Needs more RAM

From packets to insights: examples for HTTP and wireless stats

I convert kernel hooks and XDP stats into short, actionable recipes for HTTP tracing and wireless field checks. The goal: events that a user-space tool can parse fast and reliably.

Syscall hooks for HTTP flows

Hook accept4, read, write, and close with entry/exit probes. Cache args on entry in a map keyed by pid+fd. Read that map at return to confirm success and collect context.

Perf buffers, maps, and truncation

Emit two perf buffer streams: socket_open events and socket_data events. Cap payload bytes to a truncation threshold—example: MAX_MSG_SIZE = 30 * 1024.

Send metadata: total size, offset, direction.
Tag connection IDs with pid, fd, timestamp to avoid collisions.
Reconstruct large messages in user space from multiple events.

XDP stats in practice

Use a small struct record: rx_packets, rx_bytes, and last_ts. Poll from user space at a steady interval and compute pps to smooth jitter.

Field scenarios: co-channel interference and fading

Run iperf3 while bringing up an overlapping AP to test co-channel interference. Log traffic counters and information over time and watch throughput drop as collisions rise.

Test fading by changing distance and adding occluders; record packets, bytes, and retransmits. Note: the linux kernel XDP path does not expose SNR or signal strength—rely on driver counters for RF info.

Example	Function	Result
accept4/read/write/close	entry/exit maps	Reconstruct HTTP flows
perf buffers	events stream	Low-overhead user delivery
XDP counters	polling function	rx_bytes, rx_packets, pps

Next steps, reliability tips, and performance notes

I keep this pragmatic: set budgets, then fail gracefully. Cap perf buffer sizes and drop payloads under load so performance beats perfect fidelity on small routers.

Pin maps in bpffs so a restart keeps state. Clean pins on uninstall to avoid stale files. Set polling at 200–500 ms to balance CPU and monitoring fidelity.

Test driver vs generic XDP modes and disable NIC offloads if counts lie. Add watchdogs: try one re-attach, then back off to avoid crash loops.

Handle backpressure: drop data events, keep control events. Prefer short ebpf code paths in the kernel and push heavy parsing to user space.

For further diagnostics and an example on handling drops, see monitor drops with eBPF.

FAQ

What kernel and toolchain versions do I need before starting on OpenWRT?

You need a Linux kernel that supports BPF core relocations and the modern helper set — 5.4+ is a safe baseline, though newer kernels are better for features. Use the OpenWrt SDK or a matching toolchain that targets your router CPU (mipsel, arm, aarch64, or x86). Match kernel headers to the running kernel exactly; mismatches break libbpf and bpftool builds.

Which build dependencies are required to compile kernel and user-space BPF code?

Install clang/LLVM, elfutils, libelf, make, gcc, and pkg-config on the build host. For libbpf-based loaders, clone libbpf and build its headers. For BCC-based tools, add Python and BCC dependencies. Ensure bpftool is built from the same kernel tree or from iproute2 with BPF enabled.

What kernel options must be enabled to run BPF programs on my device?

Enable CONFIG_BPF, CONFIG_BPF_SYSCALL, CONFIG_BPF_JIT if you want JIT, and the networking hooks: CONFIG_BPF_STREAM_PARSER, CONFIG_BPF_LSM if using LSM, plus XDP (CONFIG_XDP_SOCKETS/CONFIG_AF_XDP) for early packet processing. Also enable CONFIG_NET and cls_bpf/act_bpf for TC hooks.

How do I build and flash firmware with the right kernel support and confirm BPF is available?

Add the kernel config fragments to your OpenWrt build config, compile the firmware, and flash using sysupgrade or the device web UI. After boot, verify with bpftool version and bpftool feature show. Check /sys/kernel/debug/tracing and bpffs mounts for program and map inspection.

What common build pitfalls should I watch for (headers, endianness, architecture)?

The top issues: using mismatched kernel headers, compiling 64-bit maps for a 32-bit router, and wrong endianness handling in packet parsing. Always build BPF objects with the same arch target and use clang -target when cross-compiling. Verify structure packing and ntohs/ntohl usage in BPF code.

What’s the quickest practical method to capture traffic on a router with limited resources?

Use XDP for minimal overhead: attach a small XDP program via iproute2 or xdp-tools that increments maps and samples headers. Push only metadata and truncated payloads into a perf buffer or ringmap to user space. That avoids heavy meter or socket layers and keeps CPU use low.

How can I debug code on-device if I can’t run heavy tooling?

Use bpf_trace_printk sparingly to write events to debugfs and collect them for post-processing. Better: emit compact events into a perf buffer and poll from a lightweight user-space loader. Keep print sizes small — large prints inflate CPU and trace output.

When should I choose BCC versus libbpf for loaders and maps?

Use BCC for fast prototyping and Python-based analysis; it’s higher-level and easier for quick tests. Choose libbpf for production: it’s leaner, supports CO-RE (BPF core relocations), and gives better performance and deterministic resource use on embedded routers.

How do I implement syscall hooks to track HTTP flows like accept4, read, write, and close?

Trace the socket lifecycle with kprobes or newer fentry hooks for accept4; use tracepoints or kprobes on read/write to sample headers. Correlate socket fd to tuple using maps keyed by pid/tgid and socket pointer. Clean up maps on close to avoid leaks.

What’s the right strategy for sending payload bytes to user space without overloading the router?

Truncate payloads at a small byte limit per event (for example 64 bytes). Use per-CPU ring buffers or perf buffers and implement back-pressure: drop or aggregate when buffers fill. Store counters and sketches in maps and only send full payloads for sampled flows.

How do I expose XDP stats like rx_packets, rx_bytes, and pps to a user-space monitor?

Maintain per-interface or per-CPU maps in the XDP program (array or percpu_hash). Periodically poll maps from a user-space loader and compute deltas to get pps. Serialize stats via unix sockets, HTTP endpoints, or simple sysfs entries for lightweight monitoring.

How can I detect wireless issues like co-channel interference or channel fading with kernel-level tools?

Combine radiotap/IEEE 802.11 header sampling in an eBPF probe with driver stats (RSSI, retries) exposed via nl80211. Aggregate metrics in maps and run churn analysis: rising retries with stable RSSI suggests interference; falling RSSI hints at fading. Feed these signals to alerts or adaptive channel-selection logic.

What’s a robust user-space loader approach for attaching XDP programs and polling stats?

Use a small libbpf-based loader that: verifies kernel features, loads CO-RE-built objects, pins maps under /sys/fs/bpf, attaches XDP to a netdev, and spawns a tight poller to read maps and perf buffers. Add graceful detach and map cleanup on SIGINT.

Any tips to improve reliability and performance on low-end routers?

Keep BPF programs tiny — favor metadata over full payloads. Use per-CPU maps to avoid contention. Rate-limit user-space events and rely on in-kernel counters for high-speed metrics. Test under load and add fallbacks to software filtering if CPU crosses thresholds.

What tools and helpers should I include in my environment for development and troubleshooting?

Keep bpftool, clang/llvm, libbpf, iproute2 with XDP support, tc, and strace for user-space debugging. Use wireshark/tcpdump on a mirror port for ground truth. Add simple dashboards that read pinned maps to visualize real-time metrics.

Tagged eBPF-based packet capturing, Enhanced Berkeley Packet Filter (eBPF), Network performance optimization, OpenWRT network monitoring, OpenWRT traffic analysis

WilliamPatterson

I'm a lifelong tech enthusiast who is particularly passionate about Linux and open-source software. My journey into the world of technology began in college, when I first discovered the power of open-source communities. I graduated with a degree in Computer Science, having spent countless hours tinkering with Linux distributions and learning about the Linux Kernel.