How missIndex Works in GLSL traceRayEXT

In hardware-accelerated ray tracing using GLSL, the missIndex parameter in the traceRayEXT function specifies which miss shader to execute when a ray fails to intersect any geometry within an acceleration structure. This mechanism enables developers to branch their rendering logic dynamically based on ray type—such as primary camera rays, shadow rays, or ambient occlusion queries—by directly referencing different entries in the Shader Binding Table (SBT).

The Role of missIndex in Ray Traversal

When invoking traceRayEXT inside a ray generation or closest hit shader, the function takes several parameters defining the ray's origin, direction, range, and traversal behavior. If the ray traverses the scene and finds no valid intersection between the minimum and maximum travel distances (tMin and tMax), hardware execution transitions immediately to a miss shader.

Because a modern ray tracing pipeline often requires distinct fallback behaviors, the pipeline layout allows multiple miss shaders to be compiled simultaneously. The missIndex parameter acts as an unsigned integer index that tells the ray tracing pipeline precisely which miss shader in the pipeline should handle the miss event for that specific ray.

Interaction with the Shader Binding Table (SBT)

The ray tracing hardware locates the target miss shader using the Shader Binding Table (SBT), which is a GPU buffer allocated by the application containing shader handles and optional local parameters.

When a ray misses, the device determines the memory address of the target miss shader record using the following calculation:

\[\text{ShaderRecordAddress} = \text{MissSBT.deviceAddress} + (\text{missIndex} \times \text{MissSBT.stride})\]

  • MissSBT.deviceAddress: The base GPU address of the miss region in the SBT.
  • MissSBT.stride: The size of each miss record in bytes.
  • missIndex: The value passed directly into traceRayEXT.

By incrementing missIndex, you index sequentially through the array of miss shader records specified in your dispatch command.

Practical Applications

Different rendering passes require unique behaviors when a ray escapes the scene:

  • Primary and Reflection Rays: Typically bound to index 0, directing execution to a miss shader that samples a high-dynamic-range (HDR) environment map or renders an analytical sky model to populate the ray payload with background color.
  • Shadow Rays: Typically bound to index 1. When casting a visibility or shadow ray, hitting empty space means the light source is unoccluded. The corresponding miss shader can be extremely lightweight, doing nothing more than writing a boolean flag (rayHit = false) to the payload.
  • Ambient Occlusion (AO): Can use its own index or share the shadow miss shader to indicate that the sample direction is unoccluded, contributing to full ambient lighting.

Implementation Considerations

The missIndex must strictly remain within the bounds of the miss shader count established when binding the SBT during the pipeline setup. Supplying a missIndex that computes an address past the allocated miss buffer range results in undefined behavior, typically manifesting as device loss, memory corruption, or visual artifacts.