You want clear backtraces and fast troubleshooting, so I show how to compile kernel with debug symbols and turn cryptic traces into actionable clues.

I’ve guided many engineers through building a linux kernel that keeps rich DWARF info, frame pointers, and module matches—so GDB and SystemTap find meaningful names instead of hex addresses.

We’ll cover exact config flags, two build paths (manual and Buildroot), and a quick QEMU/KVM loop to boot and attach a debugger. I point out architecture nuances and size vs. speed trade-offs.

By the end you’ll have a repeatable workflow to produce vmlinux, load module data at runtime, and verify your running linux image matches its debug info—so troubleshooting feels predictable, not painful.

Key Takeaways

• Builds with DWARF and frame pointers give readable backtraces for real-world debugging.
• Two practical paths: manual source builds for control, Buildroot for automation.
• Use a fast QEMU/KVM loop to boot and attach GDB for quick iteration.
• Verify module and vmlinux matches to avoid missing info during probes.
• Balance symbol size against performance—strip only what you can afford.

Why kernel debug symbols matter for Linux kernel debugging today

When addresses become names and lines, your troubleshooting shifts from guessing to solving. Enabling full symbol data turns raw addresses into function names, file:line and readable variables. That change makes every backtrace meaningful.

I recommend building with CONFIG_GDB_SCRIPTS enabled and leaving CONFIG_DEBUG_INFO_REDUCED off. When supported, enable CONFIG_FRAME_POINTER — it produces far clearer backtraces across architectures.

GDB auto-loads the Python helpers in vmlinux-gdb.py and gives us commands like lx-symbols and lx-dmesg. These helpers load module data, read kernel logs, and expose per-CPU state. They are hard to use without good symbol info.

Keep a symbolized vmlinux for every debug build as a default practice. It costs disk space, but saves hours during live triage. Tools such as SystemTap, perf, and BPF workflows also rely on precise mappings between addresses and source to resolve page offsets and types.

How to compile kernel with debug symbols on Linux

I’ll walk you through the practical steps to prepare toolchains and configs so your linux kernel build produces usable DWARF and module data.

Prerequisites: Install a recent GCC or Clang, make, and the usual build-essential packages. Lay out the kernel source tree in a single workspace so vmlinux, bzImage, and .ko modules build cleanly.

Enable the right options

Open menuconfig and enable CONFIG_DEBUG_KERNEL and CONFIG_DEBUG_INFO so DWARF lands in both the main image and every module. Turn on CONFIG_GDB_SCRIPTS and keep CONFIG_DEBUG_INFO_REDUCED off for full type data.

Frame pointers and notes

Where the architecture permits, enable CONFIG_FRAME_POINTER. Frame pointers stabilize backtraces and make stepping in GDB far easier during live sessions.

• Standard outputs: vmlinux (uncompressed DWARF), bzImage (bootable), .ko modules (carry DWARF).
• Verify cross-toolchains when targeting other arches — mismatches cause opaque failures.
• Keep a debug defconfig as the default for iterative hacking and a lean release config for production.

Expect larger files and longer links—those trade-offs cut troubleshooting time drastically when you can see names and lines instead of addresses.

Set up a debuggable target with QEMU/KVM and GDB

I start by launching a virtual target that exposes a GDB stub so I can inspect a running system instantly. This lets me pause the CPU, set breakpoints, and step at source level.

Boot options: QEMU can boot a build directly using -kernel and -append for fast iteration. When I need a full userspace and modules, I install the image inside a guest disk instead.

Starting QEMU and the GDB stub

I enable the stub with -s (TCP port 1234) so the VM waits for a debugger. I usually add KVM for speed, set memory size, and forward SSH ports for convenience.

Attach GDB and load symbols

On the host I run gdb vmlinux, and if auto-load is restricted I add an auto-load safe path in ~/.gdbinit. Then I attach with target remote :1234.

After attach, I run lx-symbols so GDB loads module symbols as modules appear. Use lx-dmesg, $lx_current(), and $lx_per_cpu() to inspect live state.

• I verify CONFIG_GDB_SCRIPTS is enabled and CONFIG_DEBUG_INFO_REDUCED is off so types are complete.
• Common commands: continue, step, finish, and pending breakpoints for drivers that load later.
• For repeatable work I boot from snapshot-capable images and restore known states between runs.

Using Buildroot to automate a debug-friendly kernel and rootfs

A single Buildroot run can create a full debugging target — toolchain, minimal userspace, and kernel artifacts ready for QEMU. I use this flow when I want a reproducible sandbox fast.

Start by running make qemu_x86_64_defconfig and then make menuconfig. Toggle BR2_ENABLE_DEBUG so every package keeps package-level symbols. Add OpenSSH and choose an ext4 root filesystem (BR2_TARGET_ROOTFS_EXT2_4) for convenient editing and logs.

Pick versions and kernel options

In Toolchain and Kernel menus set the linux version and matching headers so build and runtime agree. Open the linux menu and enable CONFIG_DEBUG_KERNEL, DEBUG_INFO, and FRAME_POINTER to get a vmlinux with full DWARF, a bzImage, and module artifacts that GDB can use.

Key outputs and next steps

After make finishes, you’ll find the source under output/build/linux-…. The raw image is output/build/linux-…/vmlinux. Compressed boot images sit in output/images/bzImage and the rootfs at output/images/rootfs.ext4.

Boot QEMU using -s to expose gdbserver on port 1234 and add networking flags: -net nic,model=virtio -net user,hostfwd=tcp::5555-:22. SSH to host port 5555 to run commands and iterate. Use QEMU savevm/loadvm via the monitor socket to snapshot states during iterative hacking.

Troubleshooting missing module debuginfo and SystemTap warnings

I often see SystemTap print messages like cannot find module nfs debuginfo. Those warnings usually mean either the modules lack DWARF or the tool cannot locate installed symbols.

Start by confirming the build enabled CONFIG_DEBUG_INFO for both the core image and modules. If modules were built without DWARF, SystemTap and GDB cannot resolve names or types during debugging.

Ensuring modules are built with DWARF and installed symbols are findable

• Check module files on disk with readelf -wi module.ko to see if DWARF sections exist.
• Verify the install path used by your build matches where tools search for symbol files.
• If the .ko lacks DWARF, rebuild using a default debug-friendly config that enables module info.

Matching running kernel, module versions, and symbol paths

Even a minor version mismatch will stop symbol resolution. I check the running version via uname -r and compare it to the built artifacts’ version string.

For GDB, I point vmlinux at the same build and run lx-symbols so module name matches load paths. After aligning versions and paths, SystemTap warnings usually vanish and my debugging becomes predictable again.

Ship-ready checks and performance-minded optimizations

I keep release builds lean and retain a full offline vmlinux and symbol archive for postmortems. This splits a production kernel that’s stripped for speed from the artifacts we need to decode crashes later.

I review config options and module lists to avoid shipping costly debug-only features. Frame pointers and heavy tracing get toggled off if performance is tight.

For incident response I store the exact vmlinux, System.map, and a symbol archive per build target and linux kernel version. That makes replaying traces and matching addresses fast.

When storage matters, I use split DWARF or external debug packages so runtime stays slim and developers keep full data offline.

I automate this in CI: a default artifact set, a QEMU boot smoke-test, and a simple GDB attach. That keeps our security posture solid while making kernel debugging one command away during real incidents.