The Linux process life cycle runs through a handful of real kernel states, not the tidy boxes on a textbook poster. A process is born when fork() clones its parent, swaps in new code with execve(), then cycles through running (R), interruptible or uninterruptible sleep (S or D), and stopped (T). It ends by calling exit(), and its parent reaps the result with wait(). You diagnose all of it by reading /proc and strace output, not by memorizing a chart. Map a stuck process to one of those states and say why it's there, or you don't understand the lifecycle yet.

Last updated: 2026-07-21

Here's the workflow I actually use: identify, confirm impact, inspect state, trace parent and child, then act. I'll also flag where people get burned – killing the wrong PID, mistaking zombies for active tasks, and chasing a D-state that won't yield. Those three mistakes cause most of the 2 a.m. damage.

From a file on disk to a running process

The shell doesn't "start" a program in one step. It calls fork() to clone itself, then the child calls execve() to replace its own memory with the new program. The parent stays as the controller. The child becomes the running instance with a fresh PID.

Watch it with ps -f so you see PID and PPID side by side. PID is the child's ID. PPID names who spawned it. If a parent keeps respawning children, you'll see repeated entries with the same PPID, and that tells you not to kill blindly. Use pstree or ps -ef to map the whole tree fast.

A program on disk is inert. It becomes a process only after the kernel builds the execution context and the scheduler hands it CPU time. That shift, from file to running task, is where nearly all troubleshooting starts. Everything after it is the kernel moving one task between states and you trying to read which state it's in.

The state letters that actually matter

Textbook diagrams say new, ready, running, waiting, terminated. Linux uses different names, and the letter in the state column is what you actually read. Learn the mapping and the diagram stops lying to you.

R does not mean "on CPU right now." It means ready. Under high load a runnable task can sit queued while something else holds the core, which is why load average and CPU usage are not the same number. D is the one that scares people, and it should: the process is blocked inside a kernel call and will not answer SIGKILL until the I/O returns.

How do processes move between states, and why do they get stuck?

Stop guessing at a stuck process and read what it's blocked on. Run cat /proc/[pid]/status and look at the State line, then read /proc/[pid]/wchan (or the WCHAN column in ps -eo pid,stat,wchan,cmd). That tells you the kernel function the task is sleeping in. A name like nfs_wait or io_schedule is your answer, not a mystery.

Most transitions are boring. A task runs, blocks on a read, drops to S, gets woken when data arrives, and goes back to R. The interesting failures are the ones that don't come back. A D-state on an NFS mount that went away will hang there until the network heals or the box reboots. No signal helps, because the kernel deliberately refuses to interrupt that wait.

For a task stuck in D, strace -p [pid] usually hangs too, because the process is inside the syscall you'd be tracing. That's a diagnosis, not a dead end. Check /proc/[pid]/stack for the kernel backtrace and go fix the storage or driver underneath. The process is a symptom, not the disease.

Foreground and background at the kernel level

The & operator is the least interesting part of job control. What actually happens is process groups and sessions. Your terminal has one foreground process group that owns the terminal. Only that group can read from it, and by default only it should write to it.

When a background job tries to read the terminal, the kernel sends it SIGTTIN and stops it. Write attempts can trigger SIGTTOU the same way. That's why a backgrounded interactive command freezes instead of stealing your keyboard. The letter you'll see is T, stopped, and jobs shows it in the current shell.

• Start a job in the background with command &, then confirm it with jobs.
• Move a stopped job to the foreground with fg %1, or resume it in the background with bg %1.
• Suspend the foreground job with Ctrl+Z, which sends SIGTSTP and drops it to T.

nohup solves a different problem. When you log out, the kernel sends SIGHUP to your session's processes, and most die. nohup makes a process immune to SIGHUP, so it keeps running after you close the terminal. Reach for it when you want work to outlive the login, not just move off the prompt. For the deeper details of which signal does what, see my walkthrough on handling signals in Linux.

How do you monitor process status without getting fooled by ps?

Start with ps aux for a full-system snapshot, but know what it is: one instant, frozen. Read three columns first under pressure – user, %CPU, and %MEM – then confirm the full command before you touch anything. Use ps -f when you need parent and child context, because it prints PPID next to PID.

ps lies in a specific way. The %CPU it shows is an average over the process's whole lifetime, not what's happening now. A process that pegged a core for an hour then went idle still shows a fat number. For live behavior, use top or htop and watch it move.

In top, the S column is the one I read first, because it tells me whether to wait, signal, or dig. pgrep -f service-name beats grepping ps output for long command lines. My fast path: pgrep to get the PID, ps -f to confirm owner and full command, then top to watch it before I send a signal. Never kill from a guess.

When sleeping is fine and when it isn't

Sleeping is normal, and most of the time it means the process is healthy and waiting on something. S, interruptible sleep, is a task parked on a socket read, a select, or a lock. It will wake the instant its event fires, and it answers signals immediately. A daemon sitting in S is doing exactly what a well-behaved Linux daemon should.

D, uninterruptible sleep, is the one worth worrying about. The task is blocked inside the kernel and cannot be interrupted, not even by SIGKILL, because waking it mid-syscall could corrupt kernel state. A few processes flickering through D is fine. A pile of them stuck in D means storage or network is not answering.

To find what a sleeping process waits on, read its WCHAN. Run ps -eo pid,stat,wchan:20,cmd and match the function name to the subsystem. pipe_read is a pipe with no writer. io_schedule is block I/O. If half your processes share one WCHAN pointing at a filesystem, that's your suspect, and killing the processes won't fix it.

Stopping and killing a process the right way

Send SIGTERM first, always. It's signal 15, the polite request, and a well-written program catches it to flush buffers and clean up. Plain kill [pid] sends SIGTERM. Give it a couple of seconds before you escalate.

SIGKILL, signal 9, is the hammer. The kernel destroys the process with no chance to clean up, which means half-written files and orphaned locks. Reach for it only when SIGTERM gets ignored. SIGSTOP (signal 19) is different again: it freezes the process without killing it, and SIGCONT resumes it. Neither SIGKILL nor SIGSTOP can be caught or blocked, by design.

• pkill -f service-name and killall name signal by name, which is handy and dangerous. Confirm with pgrep -f first so you don't hit a process you didn't mean to.
• SIGKILL will not clear a D-state. The signal queues and does nothing until the I/O returns. Fix the storage instead.

Zombies confuse people because you can't kill something already dead. A Z entry is a child that exited but whose parent never collected the result. When a process ends, it can call _exit() with an integer argument to report an exit code to its parent, and the parent is supposed to read it with wait(). Until it does, the kernel keeps the entry so the exit code isn't lost. To clear zombies, signal the parent to reap or restart it. If the parent dies first, the orphaned child is re-parented to PID 1, the init process, which adopts orphaned processes and reaps them so the table stays clean. Check the last exit code of your own commands with echo $?.

Process migration means two different things

"Migration" gets used for two completely different operations, so pin down which one you mean before you touch anything. The everyday kind is the scheduler moving a task from one CPU core to another to balance load. It happens constantly and needs no help from you.

If you want a process pinned to specific cores, use CPU affinity. taskset -pc 0-3 [pid] restricts a running process to cores 0 through 3, and taskset -c 2 command launches one there. Pinning helps for cache locality and jitter-sensitive work, but the scheduler is usually smarter than you, so measure before you pin. Read the actual mask with taskset -p [pid].

The second kind of migration moves a running process, with its memory and open file descriptors, off one machine and onto another. Linux does this through checkpoint and restore, and the tool is CRIU (Checkpoint/Restore in Userspace). It freezes a process tree, dumps its state to disk, and restores it later or elsewhere. Container platforms use this for live migration and fast startup.

Moving a running process with CRIU

The checkpoint and restore path is a deliberate operation, not a background nicety. You dump a process tree with criu dump, then bring it back with criu restore, on the same host or a different one. The state on disk holds the process memory, its open file descriptors, and its place in the tree.

Where it bites is the constraints most guides skip. Open sockets, PID reuse on the target, and mismatched kernel versions between hosts all break a restore. If a container won't come back, check those three before you blame the tool. Scheduler migration is automatic and cheap; CRIU migration you plan for. Mislabeling one as the other sends people down the wrong path.

How do you manage the lifecycle in production?

Manage the lifecycle the same way you debug anything on Linux: read the state before you act. My loop is identify, confirm impact, inspect state, trace parent and child, then act. Skipping the inspect step is how people kill the wrong PID and page themselves twice in one night.

When a service misbehaves, start from the process table and work down. systemctl status and journalctl -u service tell you restart timing and the last error. If a unit keeps respawning, the fix is almost never a bigger hammer on the child. It's in the logs and the exit code. Read the exact message instead of scrolling past it.

The habit that saves you is refusing to treat a symptom as the cause. A D-state process is downstream of a storage problem. A zombie pile is a parent that never reaps. A CPU-pinned service that still stalls is waiting on a lock, not a core. Trace it to the real actor, fix that one thing, and confirm the state changed. That's the difference between a fix that holds and one that comes back at 2 a.m. If open handles are part of the mess, my notes on managing file descriptors cover where those leak.

FAQ

How do I tell a hung process from a busy one?

Read the state letter and the WCHAN, not the CPU number. A busy process sits in R and burns CPU you can watch move in top. A hung one usually sits in D or S with a WCHAN pointing at whatever it's blocked on. Match that function name to a subsystem and you know whether it's waiting on disk, network, or a lock.

Why won't kill -9 stop my process?

Because it's in uninterruptible sleep. SIGKILL can't be delivered to a task blocked inside a kernel syscall, so the signal queues and waits. The process only dies once the I/O it's stuck on returns or times out. Go fix the storage or mount underneath; the process will clear on its own once the call completes.

What's the difference between a zombie and an orphan?

A zombie has already exited and is waiting for its parent to collect the exit status, so it holds a slot in the process table but uses no CPU or memory. An orphan is still alive, but its parent died first. The kernel re-parents orphans to PID 1, which reaps them when they eventually exit. One is a bookkeeping leftover; the other is a live process that just changed parents.

How do I keep a job running after I log out?

Launch it with nohup command &, which shields it from the SIGHUP the kernel sends at logout. For anything you'll manage long-term, though, write a proper systemd unit instead. A unit gives you restart policy, logging through journalctl, and a clean stop, none of which a bare nohup gives you.

Where does the kernel keep live process data?

In /proc, one directory per PID. cat /proc/[pid]/status shows the state, memory, and owner; /proc/[pid]/wchan shows what it's sleeping on; /proc/[pid]/stack gives the kernel backtrace for a stuck task. Tools like ps and top just read and format this for you, so going straight to /proc gets you the truth without the tool's averaging or formatting in the way.

Related: 6 Linux IPC Mechanisms: When to Use Each

Related: Fork vs. Vfork: How Memory Sharing Changes Process Safety