GLSL Mesh Shader layout(points) out for Particles

This article examines the behavior, configuration, and execution model of using layout(points) out in GLSL mesh shaders to generate particle systems. In a mesh shading pipeline, designating points as the output topology shifts the stage from traditional triangle assembly to independent point primitive generation. This allows a single compute-like workgroup to simulate particle dynamics, control primitive topology, set point attributes, and output rasterization-ready point sprites directly in hardware without intermediate buffer roundtrips.

Topology and Indexing Behavior

When using layout(points) out; in a GLSL mesh shader (using GL_EXT_mesh_shader or GL_NV_mesh_shader), the primitive assembler interprets the output as discrete point primitives. Unlike triangles, which require three vertex indices per primitive, points operate on a strict 1:1 relationship between primitives and vertices.

You must declare the maximum output allocations:

layout(local_size_x = 32) in;
layout(points, max_vertices = 32, max_primitives = 32) out;

In standard implementations:

  • The number of emitted primitives matches the number of active particles emitted by the thread group.
  • The primitive index buffer maps directly to vertex indices. For GL_EXT_mesh_shader, you assign indices to gl_PrimitivePointIndicesEXT[i]. For particle workloads where vertex i corresponds directly to point i, thread i simply sets gl_PrimitivePointIndicesEXT[i] = i.
  • The workgroup must explicitly define the output counts using SetMeshOutputsEXT(numVertices, numPrimitives); before writing to indices or per-primitive outputs.

Per-Vertex and Per-Primitive Attribute Output

Once output counts are declared, threads in the workgroup write per-particle parameters to the built-in and user-defined output arrays:

  • Position: Set via gl_MeshVerticesEXT[i].gl_Position. This represents the center of the particle in clip space.
  • Point Size: Set via gl_MeshVerticesEXT[i].gl_PointSize. The rasterizer reads this value to expand the single vertex into a screen-aligned square during rasterization.
  • Attributes: Custom vertex outputs (such as particle color, UV offsets, or alpha values) are written via standard output arrays declared with the out qualifier. Per-primitive attributes marked with perprimitiveEXT can be used to pass uniform properties across an entire particle without replicating them per-corner, which is relevant if the downstream pipeline processes primitives individually.

Rasterization and Point Sprite Mapping

Points emitted by layout(points) out enter the fixed-function rasterizer identically to points emitted from a standard vertex shader or geometry shader:

  1. The hardware evaluates gl_Position and checks whether the point lies within the clip volume (factoring in the radius determined by gl_PointSize if conservative rasterization or point clipping rules apply).
  2. The point is expanded into a screen-aligned quad centered at gl_Position.
  3. In the fragment shader, gl_PointCoord becomes available, providing normalized [0, 1] texture coordinates across the expanded quad. This enables circular masking, soft particle falloffs, and atlas lookups without requiring the mesh shader to emit four separate vertices and two triangles per particle.

Advantages for Particle Simulation and Rendering

Using layout(points) out instead of triangles or traditional compute pipelines changes several execution characteristics:

  • Unified Simulation and Render Pass: Threads in the mesh shader can load simulation states from a storage buffer, update physics (velocity, life cycle, collision) using workgroup shared memory (shared), and immediately emit the visible points to the rasterizer, bypassing compute-to-draw barrier stalls.
  • Zero Output Amplification Overhead: Unlike geometry shaders, which suffer performance drops when amplifying points into quads, the mesh shader outputs the native point topology directly, leaving quad expansion to the rasterizer.
  • Dynamic Compaction: Dead particles can be culled within the workgroup using ballot instructions or shared memory prefix sums. If only a subset of particles in a workgroup are active, SetMeshOutputsEXT dynamically scales the output count, preventing inactive particles from reaching the rasterizer.