Executing Programs with Execve in C Child Processes

In Unix-like operating systems using the C programming language, the primary mechanism allowing a child process to run an independent binary is the fork-exec pattern, centered around the execve system call. While the fork() system call duplicates the calling process to create a child, execve() subsequently replaces that child process's entire memory space, code, and execution context with a brand-new executable. This decoupled design separates the creation of a new execution context from the loading of a program, granting developers fine-grained control over process configuration prior to executing independent binaries.

The Fork-Exec Paradigm

In C, a running program cannot directly invoke execve without destroying its own execution context. Because execve is an absolute transformation rather than a subroutine call, any code following a successful execve in the original process is permanently lost.

To overcome this, operating systems rely on the two-stage fork-exec model:

  1. Process Duplication (fork): The parent process calls fork(), prompting the operating system kernel to create a child process. At the moment of creation, the child is an almost identical duplicate of the parent, sharing open file descriptors, memory contents via copy-on-write, and environment variables.
  2. Process Replacement (execve): Inside the newly created child process, code branches based on the return value of fork() (which returns 0 inside the child). The child then invokes execve to swap out its duplicated image for the target executable.

How execve Replaces the Process Image

When execve is invoked, the kernel performs several critical operations:

  • Deallocation: The kernel releases the virtual memory space of the child process, including its text (code), data, BSS, heap, and stack segments.
  • Loading: The kernel inspects the target file specified by the path. If it is an ELF binary or an executable script (designated by a shebang line like #!/bin/sh), the kernel maps the new program's segments into the process's address space.
  • Preserving the Process ID: Crucially, execve does not generate a new Process ID (PID). The child retains its original PID and Parent Process ID (PPID).
  • Setting up the Stack: The kernel constructs a fresh user stack containing the arguments array (argv) and environment variables array (envp) passed to execve.
  • Execution Transfer: The CPU's instruction pointer is reset to the program's defined entry point (typically starting at _start, which initializes runtime libraries and eventually calls main).

The execve Interface

The execve function signature is defined in <unistd.h>:

int execve(const char *pathname, char *const argv[], char *const envp[]);
  • pathname: The absolute or relative path to a valid binary executable or script.
  • argv: A null-terminated array of string pointers representing command-line arguments. By convention, argv[0] contains the filename of the program being executed.
  • envp: A null-terminated array of key-value strings (e.g., "KEY=VALUE") passed as the environment to the new program.

File Descriptors and the O_CLOEXEC Flag

By default, file descriptors opened by the parent process remain open across the execve transition in the child. This allows the child to inherit standard streams (stdin, stdout, stderr) or redirected pipelines established before the execution call.

If specific descriptors must not leak into the new program, developers apply the FD_CLOEXEC file descriptor flag (often set via fcntl or during open using O_CLOEXEC). Descriptors flagged with close-on-exec are automatically closed by the kernel when execve successfully executes.

Return Values and Error Handling

Because a successful execve call overwrites the calling program's text segment, it never returns to the caller. Execution immediately begins in the new program.

If execve does return, the call has failed (returning -1 and setting errno). Common reasons for failure include:

  • ENOENT: The specified file path does not exist.
  • EACCES: The process lacks permission to execute the file or search one of the directories in the path.
  • ENOEXEC: The target file does not match a recognized executable format.

When an error occurs, the child process must explicitly exit (typically with _exit() or exit()) to avoid inadvertently executing remaining parent instructions.