Preventing File Race Conditions with O_CREAT and O_EXCL

In C programming, concurrently creating files without proper synchronization can introduce dangerous race conditions, specifically Time-of-Check to Time-of-Use (TOCTOU) vulnerabilities. This article explains how the operating system handles atomic file creation, why standard existence checks fail in multi-process environments, and how combining the O_CREAT and O_EXCL flags in the POSIX open() system call guarantees mutual exclusion and prevents race conditions.

The File Creation Race Condition (TOCTOU)

A common mistake in file handling is separating the existence check from the file creation step. For example, a developer might check if a file exists using access() or stat(), and if it does not exist, subsequently create it using fopen() or open() with O_CREAT:

// Vulnerable non-atomic pattern
if (access(filepath, F_OK) != 0) {
    // A context switch can happen here
    int fd = open(filepath, O_WRONLY | O_CREAT, 0644);
}

Between the call to access() and the call to open(), the operating system scheduler may suspend the executing process. During this window, an attacker or another concurrent thread can create a file at that exact path or replace it with a symbolic link pointing to a critical system file. When the original process resumes, it inadvertently opens or overwrites a file it assumes it uniquely created.

How O_CREAT and O_EXCL Enforce Atomicity

To eliminate this vulnerability, the POSIX open() system call provides the O_CREAT and O_EXCL flags, which must be combined using a bitwise OR (|).

  • O_CREAT: Instructs the kernel to create the file if it does not already exist. If used alone and the file already exists, open() simply opens the existing file.
  • O_EXCL: When used alongside O_CREAT, it specifies that the call must fail if the file already exists.

When you specify O_CREAT | O_EXCL, the kernel performs the verification of existence and the creation of the file as an atomic operation. This means that within the kernel's virtual file system (VFS) and the underlying filesystem driver, the inode lookup and inode allocation occur in an uninterruptible sequence relative to other processes accessing the same directory.

If the file exists, the call fails immediately, returns -1, and sets errno to EEXIST. No file is opened, no data is overwritten, and no race condition can occur.

Correct Implementation in C

The following example demonstrates the secure, atomic pattern for creating a file that must not already exist:

#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>

int create_unique_file(const char *filename) {
    // Open for writing, create only if it does not exist, set permissions
    int fd = open(filename, O_WRONLY | O_CREAT | O_EXCL, 0600);
    
    if (fd == -1) {
        if (errno == EEXIST) {
            fprintf(stderr, "Error: File already exists. Operation aborted.\n");
        } else {
            perror("Error opening file");
        }
        return -1;
    }

    return fd;
}

Key Considerations

  1. Permissions (mode): When using O_CREAT, always provide the third argument to open() (the file permissions, such as 0600), which is modified by the process's umask. Omitting it causes undefined behavior.
  2. Symbolic Links: If O_CREAT | O_EXCL is used and the pathname is a dangling symbolic link, the call will fail with EEXIST rather than following the link and creating the target, mitigating symlink redirection exploits.
  3. Network Filesystems (NFS): On older NFS protocols (such as NFSv2), O_EXCL support was not guaranteed to be atomic across multiple clients. Modern Linux environments using NFSv3 or later support atomic creation via the CREATE_EXCLUSIVE remote procedure call.