glTF Normal Mapping and TBN Frame Handedness

This article provides an overview of how coordinate frame handedness influences normal mapping in glTF assets. It examines the mathematical role of the tangent vector's fourth component, explains how bitangent generation dictates surface orientation, and details the visual shading artifacts that occur when handedness is improperly handled across mirrored UVs and different rendering engines.

In glTF 2.0, normal mapping relies on a local coordinate system defined by three orthogonal vectors: the Normal (N), Tangent (T), and Bitangent (B), collectively known as the TBN frame. While vertex normals define the direction perpendicular to the mesh surface, tangents define the alignment of the horizontal texture coordinate (\(U\)). To reconstruct the complete tangent space in a shader, the bitangent must also be known. Rather than storing a full 3D bitangent vector per vertex, glTF optimizes storage by defining tangents as four-component vectors (X, Y, Z, W).

The fourth component, W, represents the sign or handedness of the coordinate frame and must strictly equal either 1.0 or -1.0. The runtime shader calculates the bitangent using the cross product of the normal and tangent, scaled by this handedness value:

Bitangent = cross(Normal, Tangent.xyz) * Tangent.w

A value of 1.0 denotes a right-handed coordinate system, while -1.0 denotes a left-handed coordinate system. This scalar directly controls the direction in which the reconstructed bitangent points relative to the vertical texture coordinate (\(V\)).

The primary impact of handedness appears in models utilizing mirrored or overlapping UV layouts, a standard technique used to optimize texture resolution on symmetrical geometry. When a UV shell is mirrored across an axis, the relationship between the geometric surface and the texture coordinates flips, changing the handedness of the tangent space from right-handed to left-handed. If the Tangent.w component fails to reflect this flip, the calculated bitangent will point in the exact opposite direction of the texture's \(V\) axis.

When the bitangent vector is inverted, the tangent-to-world transformation matrix transforms the green channel (Y component) of the normal map incorrectly. Visually, this causes directional lighting on mirrored surfaces to invert along the vertical texture axis. Convex details such as rivets, ridges, or muscle definition appear concave, receiving highlights on the side facing away from the light source and casting shadows where light should fall.

To ensure consistent behavior, the glTF specification mandates that tangents follow the MikkTSpace standard conventions. Baking tools like Blender and Substance Painter compute normal maps based on this standard. If an exporter neglects the W sign, assumes a fixed handedness of 1.0, or strips tangent attributes entirely, rendering engines must infer or generate them dynamically. Any mismatch between the handedness used during normal map baking and the handedness reconstructed in the real-time engine breaks shading consistency, producing seam visibility along mirrored boundaries and inverted lighting across the mesh.