Understanding gl_RayTmaxEXT in GLSL Ray Tracing

In GLSL ray tracing pipelines using hardware-accelerated extensions, gl_RayTmaxEXT serves as a critical built-in variable that defines the maximum parametric distance a ray can travel to test for geometry intersections. This article explores how gl_RayTmaxEXT operates across different shader stages, its role in optimizing bounding volume hierarchy (BVH) traversal, and how developers leverage it to control ray bounds for techniques like shadow casting and ambient occlusion.

The Function of gl_RayTmaxEXT

In ray tracing, a ray is mathematically defined as an origin point \(O\) and a normalized direction vector \(D\), parameterized by a scalar \(t\):

\[P(t) = O + t \cdot D\]

The variable gl_RayTmaxEXT represents the upper boundary of the valid range \([t_{min}, t_{max}]\). Any potential geometry intersection occurring at a distance where \(t > \text{gl\_RayTmaxEXT}\) is automatically rejected by the ray tracing hardware or driver. This prevents unnecessary intersection evaluations and bounds the ray to a finite segment in 3D space.

Interaction Across Shader Stages

The value and mutability of gl_RayTmaxEXT depend on the active shader stage within the ray tracing pipeline:

  • Ray Generation Stage: Developers set the initial value of \(t_{max}\) directly as an argument to the traceRayEXT() function. This initial float determines the maximum distance the ray will search within the acceleration structure.
  • Intersection and Any-Hit Stages: During traversal, gl_RayTmaxEXT is accessible as a read-only variable. It reflects the distance to the closest confirmed intersection found so far (or the initial \(t_{max}\) if no closer hit has been committed). This dynamic shrinking of \(t_{max}\) ensures that any newly evaluated geometry must be closer than the current closest hit.
  • Closest-Hit Stage: Once ray traversal completes and the closest valid primitive is confirmed, gl_RayTmaxEXT holds the parametric distance \(t\) to that specific intersection point. Developers use this value to reconstruct the world-space hit position:
vec3 hitPosition = gl_WorldRayOriginEXT + gl_WorldRayDirectionEXT * gl_RayTmaxEXT;
  • Miss Stage: If no geometry intersects the ray within the interval \([t_{min}, t_{max}]\), the miss shader is executed. Here, gl_RayTmaxEXT retains the value it held when traversal terminated (typically the initial maximum distance).

Performance and Traversal Optimization

Setting an accurate gl_RayTmaxEXT is essential for GPU performance. The underlying traversal hardware uses this upper limit to prune bounding volume hierarchy nodes early:

  1. Shadow and Occlusion Rays: When testing visibility between a surface and a point light, gl_RayTmaxEXT should be set to the exact distance to the light source. Traversing past the light source wastes memory bandwidth and execution cycles on occluders that lie behind the light.
  2. Ambient Occlusion (AO): Ambient occlusion only samples geometry within a short neighborhood. Restricting gl_RayTmaxEXT to a small radius ensures that the ray terminates early, skipping large portions of the scene hierarchy.
  3. Dynamic Culling: Because the traversal engine continuously updates the internal maximum distance whenever a valid hit is accepted, subsequent BVH nodes farther away than the current gl_RayTmaxEXT are skipped automatically.