Using layout(invocations = N) in GLSL Geometry Shaders

This article explains how the layout(invocations = N) layout qualifier functions within OpenGL Shading Language (GLSL) to execute parallel geometry shader instances for a single input primitive. It covers the underlying hardware scheduling mechanism, how instances are indexed via gl_InvocationID, why this approach outperforms manual loop-based geometry amplification, and its primary applications in modern graphics pipelines.

The Invocations Layout Qualifier

Introduced in GLSL 4.0 and through the ARB_gpu_shader5 extension, the invocations layout qualifier alters the execution model of the geometry stage. It is defined in the geometry shader input declaration alongside the primitive type:

layout(triangles, invocations = 4) in;

When this qualifier is present, the GPU's primitive distributor schedules \(N\) distinct instances of the geometry shader for every single primitive that passes through the primitive assembly stage. If the qualifier is omitted, the invocation count defaults to 1.

Hardware-Level Parallel Execution

When a primitive (such as a point, line, or triangle) exits the vertex or tessellation evaluation stage, it enters the primitive setup hardware. Without instancing, that single primitive maps to a single geometry shader execution thread.

With layout(invocations = N), the hardware primitive generator multiplies the incoming primitive into \(N\) identical tasks. Rather than processing the primitive \(N\) times sequentially within a single thread, the GPU distributes these \(N\) invocations across available execution units (such as compute units or streaming multiprocessors).

Each instance runs independently as an isolated hardware thread or SIMD lane. The instances share the exact same input geometry data (gl_in[]), but they execute their instructions independently and emit vertices into separate output streams. There is no shared memory, synchronization, or communication between different invocations processing the same primitive.

Differentiating Instances with gl_InvocationID

Because every parallel instance receives the identical input array gl_in[], the shader needs a mechanism to diverge behavior across instances. GLSL provides the built-in integer variable gl_InvocationID.

  • gl_InvocationID is unique to each spawned instance of the geometry shader for a given primitive.
  • Its value ranges from 0 to N - 1, where \(N\) is the value declared in invocations.

Shaders typically use gl_InvocationID as an index into uniform arrays, uniform buffer matrices, or to write directly to built-in system values such as gl_Layer or gl_ViewportIndex.

Why Invocations Outperform In-Shader Loops

A developer could theoretically achieve multiple outputs for one primitive by writing a standard for loop inside a single geometry shader instance. However, hardware instancing via layout(invocations = N) provides significant architectural advantages:

  1. Register Pressure and Occupancy: Geometry shaders have strict hardware limits on the maximum number of emitted vertices (max_vertices). Emitting a large number of vertices from a single shader thread consumes substantial on-chip temporary register space and output buffer memory, reducing overall GPU occupancy. Splitting the workload into \(N\) smaller invocations reduces the required max_vertices per thread, keeping register pressure low and hardware utilization high.
  2. Dynamic Work Balancing: The GPU scheduler can pause, interleave, or assign parallel invocations across different execution cores as resources become available, avoiding bottlenecks caused by long-running single-threaded loops.

Primary Use Cases

The parallel invocation model is primarily used for single-pass multi-view rendering:

  • Layered Rendering (Cubemaps and Cascaded Shadow Maps): Invocations allow rendering to all six faces of a cubemap or multiple shadow cascades in a single draw call. By assigning gl_Layer = gl_InvocationID;, each parallel instance transforms the incoming primitive with a different view-projection matrix and routes the output to the corresponding layer of a 2D texture array or cubemap.
  • Viewport Arrays: Using gl_ViewportIndex = gl_InvocationID;, geometry instances can be projected simultaneously to different viewports on screen, which is common in VR rendering pipelines to draw left-eye and right-eye views concurrently.