rayQueryProceedEXT Traversal Loop Execution Flow

This article provides an overview of the execution flow of the rayQueryProceedEXT traversal loop used for inline ray tracing in GLSL compute shaders. It details how the state machine advances through bounding volume hierarchies, how the loop yields execution back to the shader for candidate evaluation, and how final committed intersections are resolved once traversal concludes.

1. Initialization Phase

Before executing the traversal loop, the ray query object must be defined and configured. The shader invokes rayQueryInitializeEXT, passing:

  • The acceleration structure (accelerationStructureEXT).
  • Ray flags (e.g., gl_RayFlagsOpaqueEXT, gl_RayFlagsTerminateOnFirstHitEXT).
  • Cull mask to filter geometry.
  • Ray parameters: origin, minimum distance (\(t_{\text{min}}\)), direction vector, and maximum distance (\(t_{\text{max}}\)).

At this stage, the internal traversal state machine is initialized at the root of the acceleration structure, but traversal has not yet started.

2. The Traversal State Loop

The traversal occurs inside a while loop driven by rayQueryProceedEXT:

while (rayQueryProceedEXT(rayQuery)) {
    // Candidate processing
}

Every time rayQueryProceedEXT is called, the GPU traversal hardware/software resumes walking down the acceleration structure:

  1. Internal Traversal: The engine tests ray intersections against internal nodes (BVH bounding boxes) and leaves (geometry primitives).
  2. Opaque Primitives: If a primitive is marked opaque (either via geometry flags or ray flags), the traversal engine records the intersection automatically, updates \(t_{\text{max}}\) (if looking for the closest hit), and continues traversal without interrupting shader execution.
  3. Candidate Yielding (return true): Traversal pauses and rayQueryProceedEXT returns true only when an intersection requires custom shader intervention. This happens in two scenarios:
    • Non-Opaque Triangles: An intersection with a non-opaque triangle occurred, requiring shader logic (such as an alpha test) to accept or discard the hit.
    • Procedural Primitives (AABBs): The ray intersected an Axis-Aligned Bounding Box representing a custom procedural shape that the shader must test analytically.
  4. Traversal Complete (return false): When all possible nodes and primitives have been evaluated, or when an early termination condition is met (such as gl_RayFlagsTerminateOnFirstHitEXT), rayQueryProceedEXT returns false, exiting the loop.

3. Candidate Processing Inside the Loop

When the loop body executes, the shader queries the candidate type using:

uint candidateType = rayQueryGetIntersectionTypeEXT(rayQuery, false);

Passing false specifies querying candidate (temporary) intersections rather than committed ones. The shader handles the candidate based on its type:

  • gl_RayQueryCandidateIntersectionTriangleEXT: The shader reads candidate attributes (e.g., UV coordinates, primitive ID) to determine opacity. If the intersection is valid, it calls:

    rayQueryConfirmIntersectionEXT(rayQuery);

    If not confirmed, the traversal engine treats the triangle as transparent and proceeds without reducing the search distance.

  • gl_RayQueryCandidateIntersectionAABBEXT: The shader evaluates intersection math against the procedural geometry within the box. If a hit occurs, the shader submits it to the traversal engine by calling:

    rayQueryGenerateIntersectionEXT(rayQuery, hitT);

Once the loop body completes, control returns to the loop header, invoking rayQueryProceedEXT again to resume hardware traversal.

4. Committed Intersection Handling

After the loop exits, traversal is complete. The ray query contains either the closest confirmed intersection or no intersection at all (a miss).

The result is retrieved by inspecting the committed hit state:

uint committedType = rayQueryGetIntersectionTypeEXT(rayQuery, true);

if (committedType == gl_RayQueryCommittedIntersectionNoneEXT) {
    // Ray missed all geometry
} else {
    // Ray hit geometry (gl_RayQueryCommittedIntersectionTriangleEXT or procedural)
    float t = rayQueryGetIntersectionTEXT(rayQuery, true);
    // Retrieve other hit properties to shade the point
}

Passing true to the query accessor functions fetches the finalized data from the committed hit record.