How gl_WorldToObjectEXT Transforms World Vectors

In GLSL ray tracing shaders, gl_WorldToObjectEXT is a built-in matrix variable that converts coordinates and directional vectors from world space into the local object space of the currently intersected instance. This article explains the underlying affine matrix structure of gl_WorldToObjectEXT, the mathematical process of transforming both positions and direction vectors, and how to properly apply this matrix in intersection and hit shaders.

The Matrix Structure of gl_WorldToObjectEXT

Provided by the GL_EXT_ray_tracing extension, gl_WorldToObjectEXT is defined as a mat4x3 in GLSL. In GLSL's column-major format, a mat4x3 consists of 4 columns and 3 rows. This 3x4 affine transformation matrix represents the exact inverse of the instance's transformation matrix (gl_ObjectToWorldEXT).

Because the matrix is affine, it encapsulates rotation, non-uniform scaling, shearing, and translation, while omitting the standard projective bottom row [0, 0, 0, 1] found in conventional 4x4 projection matrices. When performing computations, GLSL implicitly evaluates the operation as:

\[\mathbf{v}_{\text{object}} = \mathbf{M}_{\text{WorldToObject}} \times \begin{bmatrix} x \\ y \\ z \\ w \end{bmatrix}\]

Transforming Positions vs. Direction Vectors

The transformation behavior depends entirely on whether the target represents a spatial point or a direction vector, which is dictated by the homogeneous \(w\)-component.

Transforming Positions

To transform a point or position from world coordinates to local coordinates, set the homogeneous coordinate \(w\) to 1.0. This ensures that the translation column of the matrix affects the output:

vec3 worldPos = ...;
vec3 objectPos = gl_WorldToObjectEXT * vec4(worldPos, 1.0);

Mathematically, this evaluates the linear transformation (rotation and scale) on the \(xyz\) components and adds the translation offset:

\[\mathbf{p}_{\text{object}} = \mathbf{M}_{3\times3} \cdot \mathbf{p}_{\text{world}} + \mathbf{T}\]

Transforming Direction Vectors

Direction vectors (such as ray directions or velocity vectors) represent displacement without a fixed origin and should not be affected by spatial translation. To transform a direction, set \(w\) to 0.0, or multiply solely against the upper-left 3x3 submatrix:

vec3 worldDir = ...;
// Method 1: Using vec4 with w = 0.0
vec3 objectDir = gl_WorldToObjectEXT * vec4(worldDir, 0.0);

// Method 2: Extracting the 3x3 rotation/scale matrix
vec3 objectDirAlt = mat3(gl_WorldToObjectEXT) * worldDir;

Both methods cancel the translation component, preserving the relative direction and applying only the instance’s rotation and scale. Note that non-uniform scaling will alter the length of the vector, so you must re-normalize the resulting vector if a unit vector is required.

Transforming Surface Normals

Transforming surface normals requires special care. Normal vectors must remain perpendicular to the tangent plane of the surface. If the instance transformation involves non-uniform scaling, transforming a normal with gl_WorldToObjectEXT directly will distort its orthogonality.

To transform a normal from world space to object space under non-uniform scaling, multiply the world normal by the transpose of the forward transformation matrix (gl_ObjectToWorldEXT):

vec3 worldNormal = ...;
vec3 objectNormal = normalize(transpose(mat3(gl_ObjectToWorldEXT)) * worldNormal);

If the instance transform uses strictly uniform scaling and orthogonal rotation, mat3(gl_WorldToObjectEXT) * worldNormal combined with normalization is mathematically sufficient.

Practical Applications in Ray Tracing Shaders

In Vulkan ray tracing pipelines (Any-Hit, Closest-Hit, and Intersection shaders), gl_WorldToObjectEXT is essential for:

  • Procedural Primitive Intersection: Custom intersection shaders define geometries like spheres, boxes, or signed distance fields (SDFs) at the object origin. gl_WorldToObjectEXT converts world-space rays into object-space rays to simplify these analytical intersection equations.
  • Local UV and Texture Mapping: When computing procedural 3D noise or sampling 3D coordinate-dependent textures, converting the hit point back to local space ensures that textures stay pinned to the geometry regardless of how the instance moves or rotates in the world.