Understanding gl_ObjectRayDirectionEXT in GLSL

In hardware-accelerated ray tracing with GLSL, gl_ObjectRayDirectionEXT provides the direction vector of the currently traversed ray transformed into the local coordinate space of the intersected geometry instance. This article explains how the Vulkan ray tracing pipeline derives this object-space vector from world space, the mathematical implications of instance scaling on vector normalization, and how shaders use this value for efficient local-space intersection tests and geometric evaluations.

Coordinate Transformation Pipeline

When a ray is dispatched using traceRayEXT, it is defined by a world-space origin (gl_WorldRayOriginEXT) and a world-space direction (gl_WorldRayDirectionEXT). Ray traversal descends from the Top-Level Acceleration Structure (TLAS) to the Bottom-Level Acceleration Structure (BLAS). Each BLAS instance in the TLAS contains an affine \(3\times4\) transformation matrix \(M\) that places the local mesh into world space.

To evaluate geometry in its native coordinate system within intersection, any-hit, and closest-hit shaders, the ray is transformed into object space using the inverse of this instance matrix (\(M^{-1}\)). Because direction vectors represent displacement rather than position, the translational component of the matrix is discarded. The object-space direction is computed as:

\[\vec{d}_{\text{object}} = M^{-1}_{3\times3} \cdot \vec{d}_{\text{world}}\]

This resulting vector is accessible via the built-in vec3 gl_ObjectRayDirectionEXT.

Vector Magnitude and Normalization Behavior

A critical characteristic of gl_ObjectRayDirectionEXT is that it is generally not normalized, even if the original world-space ray direction was a unit vector.

When an instance undergoes scaling transformations:

  • Uniform Scaling: Scaling an instance up by a factor of \(k\) scales the object-space ray direction down by a factor of \(1/k\).
  • Non-Uniform Scaling: Anisotropic scaling skews the direction and alters its magnitude according to the inverse scale along each respective axis.

The pipeline intentionally leaves gl_ObjectRayDirectionEXT unnormalized to preserve the invariant of the ray parameter \(t\). The parametric ray equation must hold true across both coordinate spaces:

\[\vec{P}_{\text{world}} = \vec{O}_{\text{world}} + t \cdot \vec{d}_{\text{world}}\] \[\vec{P}_{\text{object}} = \vec{O}_{\text{object}} + t \cdot \vec{d}_{\text{object}}\]

Because the scalar distance parameter \(t\) (gl_RayTminEXT, gl_RayTmaxEXT, and gl_HitTEXT) remains identical in both spaces, the magnitude of \(\vec{d}_{\text{object}}\) compensates directly for the local space's scale.

Usage in Custom Intersection and Procedural Shaders

gl_ObjectRayDirectionEXT is primarily used in custom intersection shaders to evaluate procedural geometry (such as spheres, cylinders, or SDF volumes) in canonical local space:

  1. Analytical Intersections: Solving quadratic equations for shapes centered at the local origin \((0,0,0)\) avoids complex transformations of the geometry itself. The ray is tested directly against canonical models using gl_ObjectRayOriginEXT and gl_ObjectRayDirectionEXT.
  2. Validating \(t\) Values: Any hit distance \(t\) computed using this vector can be passed directly to reportIntersectionEXT without manual scale adjustments.
  3. Angular Computations: If a shader requires directional analysis—such as calculating a dot product against an object-space surface normal—the developer must explicitly normalize gl_ObjectRayDirectionEXT using normalize(gl_ObjectRayDirectionEXT) to prevent distorted lighting calculations caused by object scaling.