Using rayQueryEXT for Inline Ray Traversal in GLSL

This article explores how the rayQueryEXT object structures inline ray traversal directly inside standard GLSL compute shaders using the GL_EXT_ray_query extension. It covers the complete lifecycle of a ray query, from object initialization and iterative BVH traversal to candidate intersection handling and final committed hit retrieval, demonstrating how developers maintain full control over the execution flow without relying on separate ray tracing pipelines.

Setup and Object Declaration

To enable inline ray tracing, the shader must declare the GL_EXT_ray_query extension. A ray query is represented by an opaque object type, rayQueryEXT, which maintains the internal traversal state on the GPU.

#version 460
#extension GL_EXT_ray_query : require

layout(binding = 0, set = 0) uniform accelerationStructureEXT topLevelAS;

void main() {
    rayQueryEXT rq;
    // Traversal logic follows
}

The rayQueryEXT object can be allocated as a local variable within any execution branch of a compute shader, allowing threads to independently decide if and when to cast rays.

Initializing the Ray Query

Before traversal begins, the query object must be initialized with acceleration structure data, ray flags, culling masks, and spatial parameters via rayQueryInitializeEXT.

rayQueryInitializeEXT(
    rq,                     // rayQueryEXT object
    topLevelAS,             // Acceleration structure
    gl_RayFlagsOpaqueEXT,   // Ray flags (e.g., terminate on first hit, skip any-hit)
    0xFF,                   // Cull mask
    origin,                 // Ray origin (vec3)
    tMin,                   // Minimum ray distance (float)
    direction,              // Ray direction (vec3)
    tMax                    // Maximum ray distance (float)
);

This step sets the ray's bounds and configures traversal rules—such as forcing opaque handling or disabling specific geometry flags—directly within shader execution.

Iterative Traversal Loop

Unlike pipeline ray tracing, which dispatches separate shader stages (any-hit, closest-hit, miss), inline ray query uses an explicit while loop driven by rayQueryProceedEXT.

while (rayQueryProceedEXT(rq)) {
    if (rayQueryGetIntersectionTypeEXT(rq, false) == gl_RayQueryCandidateIntersectionTriangleEXT) {
        // Evaluate candidate hit (e.g., alpha testing)
        if (alphaPasses) {
            rayQueryConfirmIntersectionEXT(rq);
        }
    }
}

The traversal lifecycle operates as follows:

  1. Traversal Step: rayQueryProceedEXT(rq) advances the ray through the bounding volume hierarchy (BVH). It returns true if the ray encounters a candidate intersection that requires shader intervention (such as non-opaque geometry or custom procedural primitives).
  2. Candidate Evaluation: Passing false to rayQueryGetIntersectionTypeEXT queries the status of the current candidate intersection. If custom logic (such as a texture-based transparency check) accepts the hit, calling rayQueryConfirmIntersectionEXT(rq) marks it as valid.
  3. Termination: Once traversal exhausts all BVH nodes or fulfills early termination criteria (such as finding an opaque hit under gl_RayFlagsTerminateOnFirstHitEXT), rayQueryProceedEXT returns false, and the loop exits.

If the ray flags force all geometry to be opaque (gl_RayFlagsOpaqueEXT), rayQueryProceedEXT traverses to completion internally without returning true, allowing the loop condition to execute just once before exiting.

Extracting Intersection Data

Once traversal completes, the shader queries the final, committed intersection by passing true to the intersection query functions.

uint hitType = rayQueryGetIntersectionTypeEXT(rq, true);

if (hitType != gl_RayQueryCommittedIntersectionNoneEXT) {
    float hitT = rayQueryGetIntersectionTEXT(rq, true);
    int instanceID = rayQueryGetIntersectionInstanceIdEXT(rq, true);
    int primitiveID = rayQueryGetIntersectionPrimitiveIndexEXT(rq, true);
    vec2 barycentrics = rayQueryGetIntersectionBarycentricsEXT(rq, true);

    // Compute shading using retrieved parameters
} else {
    // Handle miss (e.g., sample skybox)
}

Key Structural Differences from Ray Tracing Pipelines

  • State Preservation: Because traversal occurs inline inside the compute shader thread, local registers and control flow remain intact. There is no need to pack data into ray payloads or callable shader parameters.
  • Low Overhead: Ray queries avoid the overhead of full pipeline switches, shader binding table (SBT) lookups, and dynamic shader dispatch, making them ideal for ambient occlusion, shadow rays, and simple visibility checks.
  • Control Flow Flexibility: Traversal can be nested, executed conditionally within complex divergent logic, or run multiple times within a single compute invocation.