Computing Skeletal Skinning in GLSL Mesh Shaders

This article explores how modern graphics pipelines leverage GLSL mesh shaders to execute skeletal vertex skinning directly across meshlet vertices. Instead of relying on traditional vertex shader stages, mesh shaders allow a compute-like programming model where a thread cooperative workgroup processes an entire meshlet, loads bone transform matrices, computes blend weights, transforms vertex attributes, and emits triangle primitives in a single unified stage.

The Meshlet Architecture for Skinning

In traditional rendering, skinning occurs per-vertex in isolated vertex shader invocations. With the mesh shading pipeline (GL_EXT_mesh_shader or GL_NV_mesh_shader), models are partitioned into small sub-meshes called meshlets, typically containing up to 64 vertices and 126 triangles.

Skinning within a mesh shader executes cooperatively. An entire thread workgroup is assigned to a single meshlet. Each thread in the workgroup transforms one or more vertices of that meshlet by blending joint transformation matrices according to pre-assigned vertex bone weights.

Buffer Organization

To execute skinning, the mesh shader accesses three primary data structures through Shader Storage Buffer Objects (SSBOs) or Uniform Buffer Objects (UBOs):

  1. Vertex Attribute Buffer: Stores untransformed local-space positions, normals, tangents, 4-component bone indices (typically uvec4), and 4-component bone weights (typically vec4).
  2. Meshlet Descriptor Buffer: Contains offsets and counts for the vertices and primitives belonging to each meshlet.
  3. Bone Matrix Palette: An array of mat4 transforms representing the current pose of the skeleton in global or camera-relative space.

Per-Vertex Skinning Computation

Within the mesh shader invocation, the workgroup size matches the maximum vertex count of the meshlet (for example, layout(local_size_x = 64) in;). Each thread processes its assigned vertex using gl_LocalInvocationIndex.

The skinning calculation follows standard linear blend skinning (LBS) mathematics:

\[M_{skin} = \sum_{i=0}^{n-1} w_i \cdot B_{idx_i}\]

Where:

  • \(w_i\) is the blend weight of the \(i\)-th bone.
  • \(B_{idx_i}\) is the transformation matrix of the bone at index \(idx_i\).
  • \(n\) is typically 4 (maximum influences per vertex).

Implementation in GLSL

The following snippet demonstrates the standard skinning implementation within an EXT_mesh_shader routine:

#version 460
#extension GL_EXT_mesh_shader : require

layout(local_size_x = 64) in;
layout(triangles, max_vertices = 64, max_primitives = 126) out;

struct Vertex {
    vec4 position;
    vec3 normal;
    uvec4 boneIndices;
    vec4 boneWeights;
};

layout(std430, binding = 0) readonly buffer VertexBuffer {
    Vertex vertices[];
};

layout(std430, binding = 1) readonly buffer BoneMatrices {
    mat4 bones[];
};

layout(std430, binding = 2) readonly buffer MeshletVertexOffsets {
    uint meshletVertexOffsets[];
};

uniform mat4 u_ViewProjection;

void main() {
    uint meshletID = gl_WorkGroupID.x;
    uint localID = gl_LocalInvocationIndex;
    
    // Total vertices to output for this meshlet (e.g., 64 or less)
    SetMeshOutputsEXT(64, 126);

    if (localID < 64) {
        uint globalVertexIndex = meshletVertexOffsets[meshletID] + localID;
        Vertex v = vertices[globalVertexIndex];

        // Linear blend skinning matrix accumulation
        mat4 skinMatrix = 
            v.boneWeights.x * bones[v.boneIndices.x] +
            v.boneWeights.y * bones[v.boneIndices.y] +
            v.boneWeights.z * bones[v.boneIndices.z] +
            v.boneWeights.w * bones[v.boneIndices.w];

        // Transform local position and normal
        vec4 skinnedPosition = skinMatrix * v.position;
        vec3 skinnedNormal = mat3(skinMatrix) * v.normal;

        // Output transformed vertex to screen space
        gl_MeshVerticesEXT[localID].gl_Position = u_ViewProjection * skinnedPosition;
    }

    // Primitive connectivity setup occurs here concurrently
}

Shared Memory and Cache Optimization

Direct skinning in mesh shaders provides substantial caching benefits over vertex shaders:

  • Matrix Caching: If many vertices within a meshlet share the same subset of bones, the relevant bone matrices can be loaded collaboratively into shared memory by the workgroup, minimizing global memory access.
  • Tight Attribute Interleaving: Position, index, and weight attributes remain local to the core executing the workgroup, reducing external memory traffic.
  • Early Pruning Integration: Combined with a task shader stage (amplification shader), skinned meshlets can be frustum-culled or LOD-selected before any skinning math takes place, avoiding vertex matrix calculations on unrendered geometry entirely.