glTF GPU Instancing Transformation Matrices Explained

This article provides an overview of how instance transformation matrices are structured and stored in glTF using the EXT_mesh_gpu_instancing extension. It explains the decomposed transformation format, how attributes are mapped within binary buffer views, and the process by which modern graphics pipelines reconstruct the final transformation matrices for GPU rendering.

Decomposed TRS Representation

Unlike systems that store complete 4x4 matrices directly in vertex buffers, the official EXT_mesh_gpu_instancing extension stores transformation data in a decomposed Translate-Rotate-Scale (TRS) format. Storing full 16-element matrices requires four consecutive vec4 vertex attribute slots per instance, consuming 64 bytes per matrix and increasing memory bandwidth usage.

To optimize storage, glTF breaks down instance transformations into individual accessor attributes:

  • TRANSLATION: Stored as a VEC3 component (float, or normalized integer formats), representing the positional offset in local space.
  • ROTATION: Stored as a VEC4 component containing a normalized unit quaternion \((x, y, z, w)\) defining orientation.
  • SCALE: Stored as a VEC3 component defining the scaling factors along the X, Y, and Z axes.

These attributes are optional; if an instance transform only requires translation, the ROTATION and SCALE attributes can be omitted entirely, adopting default identity values (no rotation and unit scale).

Binary Buffer Layout

Instance attributes point to standard glTF accessor objects, which in turn point to bufferView elements containing raw binary data.

The binary data in the underlying buffer can be organized in two primary ways:

  1. Separate Buffers (Contiguous Arrays): Each attribute has its own tightly packed array. For example, a TRANSLATION buffer view contains all \(N\) translation vectors contiguously, followed by a separate buffer view for rotations.
  2. Interleaved Buffers: All attributes for a single instance are stored together sequentially in memory. In an interleaved configuration, instance \(i\) contains translation, rotation, and scale grouped together, defined by a specific byteStride in the corresponding bufferView.

GPU Ingestion and Matrix Reconstruction

When rendering, graphics APIs (DirectX, Vulkan, WebGL, or Metal) configure these buffers as instanced vertex attribute streams with a step rate (or attribute divisor) of one per instance rather than one per vertex.

The vertex shader reads the TRANSLATION, ROTATION, and SCALE attributes per instance and dynamically reconstructs the local \(4 \times 4\) instance matrix:

  1. Scale Matrix (\(S\)): Built from the VEC3 scale vector along the diagonal.
  2. Rotation Matrix (\(R\)): Converted directly from the unit quaternion vector.
  3. Translation Matrix (\(T\)): Built from the VEC3 position vector in the fourth column.

The complete instance transform matrix is computed on the GPU via matrix multiplication (\(M_{\text{instance}} = T \times R \times S\)) and is then multiplied by the node’s global transform matrix to place each instance into world space.