Why Solid Archiving Limits 7-Zip Multithreading

Solid archiving in 7-Zip significantly improves compression ratios by treating multiple files as a single continuous data stream, but this mechanism creates inherent sequential dependencies that hinder multi-core CPU performance. Because standard multi-threading relies on breaking data into independent chunks that can be processed simultaneously, the continuous nature of a solid archive creates a fundamental conflict between maximum compression density and parallel processing efficiency.

How Solid Archiving Works

In a non-solid archive, 7-Zip compresses each file individually. If one file finishes compressing, the next file begins with a fresh compression state.

In a solid archive, 7-Zip concatenates files into one uninterrupted stream before passing them to the compression algorithm (typically LZMA or LZMA2). This approach allows the compressor to detect repeating patterns across different files. For example, if you have ten versions of a document or software source code, a solid archive references the shared text across all files, resulting in a dramatically smaller final archive size.

The Conflict Between Solid Streams and Multithreading

Multi-threaded compression requires dividing the workload across multiple CPU cores. For multiple threads to work concurrently without waiting for each other, their assigned tasks must be independent.

Solid archiving limits this parallel execution due to several key factors:

1. Sequential Dictionary Dependencies

The core of LZMA compression is a sliding dictionary—a buffer that keeps track of previously seen byte sequences to replace future occurrences with short references. In a true solid archive, compressing byte 1,000,000 relies directly on the dictionary state generated by bytes 1 through 999,999. A second CPU thread cannot compress the next segment of data accurately until the first thread has fully finished analyzing its preceding segment and generated the updated dictionary state. This sequential dependency prevents parallel execution across a single unified stream.

2. LZMA Algorithm Limitations

The original LZMA algorithm natively supports a maximum of two threads, regardless of whether solid mode is enabled. One thread runs the match finder (searching for duplicate patterns), while the other thread runs the range encoder (writing the compressed bits). Because the algorithm does not natively divide the stream into independent parallel tasks, it cannot scale across modern multi-core processors.

3. LZMA2 Block Splitting Trade-Offs

To improve multi-threading, 7-Zip introduced LZMA2 as the default format. LZMA2 achieves parallel processing by splitting the input stream into discrete chunks (chunks usually ranging from a few megabytes to several hundred megabytes) and assigning each chunk to a separate CPU thread.

However, when applied to solid archiving, this introduces a compromise:

• Dictionary Resets: Each thread compresses its chunk independently. While a thread can copy initial dictionary data from a preceding block, it cannot dynamically update against concurrent blocks.
• Reduced Efficiency: True solid continuity is interrupted at every thread boundary. To maintain solid behavior across the entire archive, 7-Zip must either serialize the process (reducing CPU utilization) or split the solid stream into chunks (reducing the compression ratio compared to a purely single-threaded solid archive).

4. Solid Block Size Constraints

7-Zip allows users to configure the "Solid Block Size." If the block size is set to a fixed limit, 7-Zip creates independent solid blocks. Threads can compress different solid blocks in parallel, but only if there are enough separate blocks available. If an archive contains fewer solid blocks than there are available CPU threads, surplus cores remain idle.

Summary

Solid archiving prioritizes space efficiency over processing speed. By requiring a continuous history of previously compressed data to eliminate cross-file redundancy, solid archives prevent 7-Zip from cleanly distributing the compression workload across multiple CPU cores without breaking the very continuity that makes solid archiving effective.