How Linux Log Rotation Works with Logrotate

Linux systems generate continuous streams of log data that can quickly consume available disk space if left unmanaged. The logrotate utility solves this problem by automating the rotation, compression, removal, and mailing of log files based on predefined schedules or size limits. This article explains the underlying mechanics of logrotate, its configuration structure, execution methods, and the primary rotation strategies used by the operating system.

Execution Mechanism: Cron and Systemd

Unlike persistent daemons, logrotate does not run continuously in the background. Instead, it is a command-line tool executed at regular intervals by the system's scheduling system:

• Cron: Traditionally, logrotate is invoked daily via a shell script located at /etc/cron.daily/logrotate.
• Systemd Timers: Modern distributions often trigger the service using logrotate.timer, which runs logrotate.service once per day.

When triggered, logrotate reads its configuration files, verifies whether any log files meet the criteria for rotation (such as age or file size), and applies the specified actions.

Configuration Architecture

The behavior of logrotate is governed by a hierarchical configuration system:

1. Global Configuration (/etc/logrotate.conf): Defines system-wide default settings, such as default rotation frequency, compression options, and retention counts. It also includes an include /etc/logrotate.d directive.
2. Application-Specific Directory (/etc/logrotate.d/): Packages installed on the system (like NGINX, Apache, or Rsyslog) place their own rotation files here. Directives defined in these files override the global defaults.

A typical configuration block defines target files and directives within curly braces:

/var/log/app/*.log {
    daily
    missingok
    rotate 7
    compress
    delaycompress
    notifempty
    create 0640 appuser appgroup
    sharedscripts
    postrotate
        systemctl reload app > /dev/null 2>&1 || true
    endscript
}

Key Directives and Behaviors

• Rotation Frequency: Driven by time (daily, weekly, monthly) or size limits (size 100M, maxsize 50M).
• Retention (rotate <count>): Specifies how many rotated log files to retain before older ones are deleted. Setting rotate 4 with weekly keeps four weeks of historical logs.
• Compression (compress / delaycompress): Compresses rotated files with gzip by default. delaycompress defers compression of the most recently rotated log until the next cycle to prevent issues with processes that still hold an open file handle.
• File Creation (create): Immediately creates a new, empty log file with specified permissions and ownership after rotating the old file.
• Error Handling (missingok / notifempty): missingok prevents errors if the log file is absent; notifempty skips rotation if the file is zero bytes.

Rotation Methods: Rename vs. Copytruncate

Applications continuously write to log files via open file descriptors. logrotate manages this using one of two primary methods:

1. Rename and Signal (Default)

1. logrotate renames app.log to app.log.1.
2. A new, empty app.log is created using the create directive.
3. The application is signaled (usually via SIGHUP or systemctl reload inside a postrotate script block) to reopen its log file and begin writing to the new app.log.

2. Copytruncate (copytruncate)

1. logrotate copies the current contents of app.log to app.log.1.
2. logrotate then truncates the original app.log in-place to zero bytes.

This approach is used for applications that cannot be easily signaled to reopen log files. However, there is a minimal risk of losing log data written between the copy and truncate steps.

State Tracking and Testing

To avoid rotating logs multiple times within a single period, logrotate tracks historical activity in a state file, typically located at /var/lib/logrotate/status or /var/lib/logrotate.status. This file records the date and time each log was last rotated.

Administrators can test configurations and verify rotation logic using two built-in flags:

• Debug Mode (logrotate -d /etc/logrotate.conf): Simulates the rotation process, outputs detailed decisions to stdout, and makes no actual changes to files.
• Force Mode (logrotate -f /etc/logrotate.conf): Forces rotation immediately, ignoring schedule constraints and updating the state file.