Group Shared Memory in DirectX Compute Shaders

This article explores how thread group shared memory (TGSM) enables high-performance inter-thread communication in DirectX DirectCompute. In compute shaders, individual threads within the same thread group often need to share intermediate calculations, aggregate values, and coordinate complex tasks. Group shared memory acts as a high-speed, on-chip software-managed cache that bypasses slower global VRAM, paired with synchronization barriers to ensure coherent and race-free data exchange between concurrent threads.

The Role of Group Shared Memory

In DirectX DirectCompute (HLSL), threads are organized into three-dimensional thread groups. While each thread has access to its own private registers and the overall global GPU memory (structured buffers, byte address buffers, and textures), direct communication between registers is impossible.

Group shared memory bridges this gap. It is allocated per thread group on physical on-chip SRAM—typically shared with the L1 cache hardware—providing latency and bandwidth orders of magnitude faster than off-chip video memory. Declared with the groupshared keyword in HLSL, this memory space is visible to every thread in the executing group, allowing them to pass data laterally during execution.

The Mechanism of Inter-Thread Communication

Inter-thread communication via shared memory follows a cooperative read-write pattern:

  1. Independent Loading and Writing: Individual threads fetch data from global buffers or compute initial intermediate values using their private registers. Each thread then writes its results to a designated index in the groupshared array, typically mapped using the thread's local identifier (SV_GroupThreadID or SV_GroupIndex).
  2. Synchronization Barriers: Because threads within a group execute in warps or wavefronts that may not run in lockstep, execution and memory hazards (such as read-after-write collisions) can occur. To safely communicate, threads must execute synchronization intrinsics:
    • GroupMemoryBarrier(): Guarantees that all pending writes to shared memory by all threads in the group are committed and visible.
    • GroupMemoryBarrierWithGroupSync(): Blocks execution until all threads in the thread group have arrived at the barrier and all shared memory accesses are complete.
  3. Cross-Thread Reading: Once the barrier is cleared, any thread can safely read data produced by any other thread in the group. This enables threads to perform operations on neighboring data without re-fetching from global memory.

Common Implementation Patterns

  • Parallel Reductions: Algorithms like calculating the minimum, maximum, or sum of an array use shared memory to combine thread outputs in a tree-based reduction. Threads iteratively add neighboring entries in shared memory, halving the active thread count at each step until one thread produces the final group result.
  • Neighborhood Caching (Tile-Based Filtering): In image processing filters (such as blurs, edge detection, or bloom), adjacent pixels overlap across processing kernels. Threads cooperatively load a tile—including its halo/border pixels—into shared memory once. Threads then read neighboring values directly from the shared cache, drastically reducing redundant global texture samples.
  • Prefix Sums (Scan Operations): Multi-pass parallel scan algorithms use shared memory to store running totals across threads, exchanging phase values between barrier syncs to achieve linear-time execution across the group.

Performance Considerations

  • Bank Conflicts: Shared memory is split into hardware memory banks (typically 32 banks of 32-bit words). If multiple threads access different addresses within the same memory bank simultaneously, accesses are serialized. Structuring data access strides to avoid bank conflicts is essential for maintaining peak bandwidth.
  • Occupancy vs. Allocation: DirectCompute allows up to 32 KB (CS 4.x) or 32 KB/64 KB (CS 5.0+) of shared memory per thread group. Allocating large amounts of shared memory per group limits the number of active thread groups a compute unit can keep in flight simultaneously, which can lower overall GPU occupancy and latency hiding capability.