Assigning gl_PrimitiveTriangleIndicesEXT in GLSL

In a GLSL mesh shader using the GL_EXT_mesh_shader extension, assigning triangle index triplets to gl_PrimitiveTriangleIndicesEXT defines the topology of the generated primitives. This process requires configuring the shader layout for triangle primitives, allocating primitive counts using SetMeshOutputsEXT, and mapping three local vertex indices via a uvec3 vector to the target primitive index within the output array.

Shader Configuration and Built-In Declaration

To output triangle primitives, the shader must enable the GL_EXT_mesh_shader extension and declare triangles in the output layout qualifier:

#version 450
#extension GL_EXT_mesh_shader : require

layout(local_size_x = 32, local_size_y = 1, local_size_z = 1) in;
layout(triangles, max_vertices = 64, max_primitives = 126) out;

When triangles is specified as the output layout, GLSL implicitly defines the built-in output array for indices as:

out uvec3 gl_PrimitiveTriangleIndicesEXT[];

Allocating Outputs

Before writing to gl_PrimitiveTriangleIndicesEXT, the workgroup must set the actual number of emitted vertices and primitives by calling SetMeshOutputsEXT. Only the first invocation typically needs to call this, or all invocations can execute it with uniform values:

SetMeshOutputsEXT(numVertices, numPrimitives);

Assigning Index Triplets

Each element in gl_PrimitiveTriangleIndicesEXT corresponds to a single output triangle. The index triplet assigned must be a uvec3, where each component refers to a 0-based local vertex index emitted within the same mesh shader workgroup (ranging from 0 to numVertices - 1).

Assignment is typically distributed across threads in the workgroup using gl_LocalInvocationIndex:

uint primIndex = gl_LocalInvocationIndex;

if (primIndex < numPrimitives) {
    uint i0 = primIndex * 3 + 0;
    uint i1 = primIndex * 3 + 1;
    uint i2 = primIndex * 3 + 2;

    gl_PrimitiveTriangleIndicesEXT[primIndex] = uvec3(i0, i1, i2);
}

Key Considerations

• Local Vertex Scope: The index values do not reference global buffers; they reference the workgroup-local vertex outputs defined in gl_MeshVerticesEXT or custom out vertex arrays.
• Winding Order: The component order inside uvec3(i0, i1, i2) determines front- and back-facing orientations based on the pipeline's configured rasterization state (such as counter-clockwise or clockwise).
• Index Bounds: The indices in each uvec3 must not exceed or equal the vertex count passed to SetMeshOutputsEXT, and the array index for gl_PrimitiveTriangleIndicesEXT must be strictly less than the specified primitive count.