How Does GPU SIMT Differ from CPU SIMD Vectorization?

Single Instruction, Multiple Threads (SIMT) on GPUs and Single Instruction, Multiple Data (SIMD) on CPUs both accelerate data-parallel workloads by executing the same operation across multiple data elements simultaneously. While they share this foundational goal, they differ significantly in their hardware abstractions, control flow handling, memory access patterns, latency-hiding strategies, and programming models. Understanding these differences helps developers choose the right architecture and write efficient parallel code.

Execution Model and Hardware Abstraction

In traditional CPU SIMD vectorization—such as x86 AVX-512 or ARM Neon—a single instruction stream operates on wide vector registers. A programmer or compiler explicitly packs \(N\) data elements into a vector register of width \(W = N \times b\) bits, where \(b\) is the bit width of each element. There is only one program counter, one call stack, and one set of scalar control registers for the entire vector.

In contrast, the GPU SIMT model exposes each data element as an independent logical thread. Each thread has its own register state, local variables, and logical program counter. The GPU hardware groups these threads into fixed-size execution units—commonly called warps (32 threads on NVIDIA GPUs) or wavefronts (64 threads on AMD GPUs)—and executes them in lockstep on a multiprocessor.

Control Flow and Branch Divergence

Handling conditional branches (if/else) is one of the sharpest distinctions between SIMT and SIMD:

  • CPU SIMD: Conditional execution requires explicit masking. The programmer or compiler must compute a bitmask, execute both branches or use masked instructions, and blend the results into the destination vector register.
  • GPU SIMT: Threads can branch independently using standard scalar control-flow syntax. When threads within the same warp take different paths—a phenomenon known as branch divergence—the hardware automatically serializes the execution of each path, using an internal active mask to disable threads not on the current path. Modern GPU architectures (such as NVIDIA Volta and later) even support independent thread scheduling, allowing threads to reconverge at finer granularities.

Memory Access Patterns

CPU SIMD instructions are heavily optimized for contiguous, aligned memory loads and stores. Accessing non-contiguous data requires slower gather/scatter instructions or manual shuffling across vector lanes.

GPU SIMT allows each thread to compute its own arbitrary memory address independently. To maintain high bandwidth, the GPU memory controller dynamically inspects the addresses generated by all threads in a warp. When threads access adjacent memory locations, the hardware merges those requests into a minimal number of wide memory transactions, a process called memory coalescing.

Latency Hiding and Scheduling

CPUs rely on large multi-level caches, branch predictors, and out-of-order execution engines to minimize the latency of a few active threads. SIMD units sit inside these complex cores and benefit from the core's ability to keep data close in cache.

GPUs take the opposite approach: they dedicate most of their silicon to arithmetic logic units (ALUs) rather than large caches or complex control logic. To hide the high latency of global memory accesses (which can take hundreds of clock cycles), GPUs rely on massive hardware multithreading. A single streaming multiprocessor maintains the register state for dozens of warps simultaneously. When one warp stalls waiting for memory, the warp scheduler switches to another ready warp in a single clock cycle with zero context-switch overhead.

Programming Model

Writing CPU SIMD code often involves either relying on compiler auto-vectorization—which can be fragile for complex loops—or writing low-level compiler intrinsics tied to a specific instruction set width (e.g., 128-bit, 256-bit, or 512-bit).

SIMT kernels (written in CUDA, HIP, or OpenCL) are written from the perspective of a single scalar thread. The hardware and runtime handle mapping those threads onto the underlying vector width, making the source code more portable across GPU generations with different core counts and execution widths.