How Does Simultaneous Multithreading Share Execution Units?

Simultaneous Multithreading (SMT) allows a single physical processor core to execute instructions from two or more independent hardware threads during the same clock cycle. By duplicating architectural state such as general-purpose registers and program counters while sharing the core's superscalar execution units, caches, and branch predictors, SMT fills idle pipeline slots caused by data dependencies or cache misses.

Architectural State Duplication vs. Shared Resources

A superscalar processor can issue and execute multiple instructions per clock cycle, provided those instructions are independent. In single-threaded execution, instruction-level parallelism (ILP) is often limited by true data dependencies, control hazards, and memory latency, leaving many execution ports idle.

To support two hardware threads on one core, the processor duplicates only the per-thread architectural state:

  • Program counters (PC)
  • Architectural register files (or thread-tagged register rename maps)
  • Control and status registers, including interrupt masks

Everything else in the out-of-order execution engine is shared. This includes the instruction fetch and decode bandwidth, the physical register file, the reorder buffer (ROB), the issue queue, and the functional units such as Arithmetic Logic Units (ALUs), Floating-Point Units (FPUs), and load/store units.

Pipeline Stages in an SMT Core

In a 2-way SMT core, instructions from Thread 0 and Thread 1 flow through the pipeline together, tagged with their respective thread IDs.

  1. Fetch and Decode: The front end alternates or dynamically prioritizes fetching instructions from either thread based on queue occupancy or stall status, feeding them into shared or partitioned decode stages.
  2. Register Renaming: Each thread's architectural registers are mapped onto a larger, shared pool of physical registers. Thread tags prevent register name collisions between threads.
  3. Dispatch and Issue: Renamed micro-operations (\(\mu\text{ops}\)) enter the issue queue. Each cycle, the scheduler selects ready \(\mu\text{ops}\) from either thread whose operands are available and dispatches them to idle execution ports.
  4. Execution and Writeback: The ALUs, FPUs, and memory pipelines execute \(\mu\text{ops}\) without needing to know which thread they belong to, writing results back to the shared physical register file.
  5. Commit (Retirement): Instructions must retire in program order per thread. The reorder buffer tracks each thread separately (either partitioned or tagged) so that an exception or branch misprediction in Thread 0 does not corrupt Thread 1.

Throughput Gains and Contention

If a single thread achieves an average Instructions Per Cycle (IPC) of \(I_1\) on a core with peak issue width \(W\), the unused execution capacity per cycle is \(W - I_1\). By introducing a second thread, SMT converts thread-level parallelism (TLP) into instruction-level parallelism, raising total core throughput:

\[I_{\text{total}} = I_0 + I_1 \le W\]

Because both threads compete for shared structures—such as L1/L2 caches, Translation Lookaside Buffers (TLBs), and execution ports—each individual thread typically runs slower than it would alone. However, combined throughput increases significantly, often by 15% to 30% for general-purpose workloads, whenever the two threads have complementary resource demands (for example, one thread waiting on memory while the other performs integer arithmetic).