Why Do Compressed Instructions Reduce I-Cache Misses?

Compressed instruction extensions reduce instruction cache misses in embedded classical computing by shrinking the memory footprint of executable code. When instructions are encoded in 16 bits instead of 32 bits, more operations fit inside each cache line and within the total capacity of the instruction cache. This higher code density increases spatial locality and reduces capacity-driven evictions, allowing the processor to fetch more useful work from fast on-chip memory rather than stalling on slower main memory.

Higher Code Density Per Cache Line

In a standard 32-bit RISC architecture, every instruction occupies 4 bytes. A typical 64-byte instruction cache line can hold 16 standard instructions. Compressed instruction extensions, such as RISC-V's C extension (RVC) or ARM's Thumb-2, encode the most frequent operations into 16-bit (2-byte) formats.

If a program achieves an average instruction size of \(2.5\) bytes through a mix of 16-bit and 32-bit instructions, the number of instructions per 64-byte cache line rises from 16 to roughly 25. When the processor fetches a single cache line on a miss, it brings in significantly more subsequent instructions, directly lowering the compulsory miss rate for sequential code execution.

Reduced Capacity Misses in Small Caches

Embedded systems operate under strict silicon area, power, and cost constraints, which limits instruction cache sizes to a few kilobytes (often 4 KB to 32 KB). When a program's working set exceeds the cache size, capacity misses occur as older instructions are evicted and later re-fetched.

Compressed instructions shrink the total binary size by roughly 25% to 35% in typical embedded workloads. A smaller working set is more likely to fit entirely within the available cache capacity:

\[\text{Effective Cache Capacity} \propto \frac{\text{Physical Cache Size}}{\text{Average Instruction Width}}\]

By reducing the average instruction width, compressed extensions effectively expand the logical capacity of the same physical SRAM array without increasing die area or static leakage power.

Fewer Conflict Misses and Better Prefetching

Beyond raw capacity, smaller code footprints reduce the address range spanned by hot loops and frequently called functions. When a loop body occupies fewer cache sets, it is less likely to collide with other active code blocks in direct-mapped or low-associativity caches, reducing conflict misses.

Additionally, because each fetched cache line contains more instructions, the instruction fetch unit spends fewer cycles waiting on cache refills. This improves the efficiency of sequential prefetchers and keeps the processor pipeline fed, which is critical for maintaining predictable performance and low energy-per-instruction in embedded classical processors.