GPU Occlusion Culling to Indirect Draw Buffers in GLSL

This article explains how GPU-driven occlusion culling pipelines evaluate object visibility and output command data directly to indirect draw buffers using GLSL compute shaders. By bypassing CPU readbacks, modern rendering engines use compute shaders to perform bounding-volume depth tests, atomically compact visible instances, and write parameters directly into GPU-accessible Shader Storage Buffer Objects (SSBOs) that back OpenGL indirect draw calls.

The Indirect Draw Data Layout

OpenGL provides indirect drawing through commands such as glDrawElementsIndirect and glMultiDrawElementsIndirect. These commands read their parameters directly from a buffer bound to GL_DRAW_INDIRECT_BUFFER. In GLSL, this buffer can be bound simultaneously as a Shader Storage Buffer Object (SSBO) using the std430 layout qualifier.

For indexed geometry, each indirect command structure in GLSL corresponds to the C equivalent DrawElementsIndirectCommand:

struct DrawElementsIndirectCommand {
    uint count;         // Number of indices to draw
    uint instanceCount; // Number of instances to draw
    uint firstIndex;    // Offset in the index buffer
    int  baseVertex;    // Value added to each index
    uint baseInstance;  // Start instance ID
};

When writing commands or instances dynamically, two SSBOs are typically defined: one holding the indirect command structures and another holding a counter or visibility array.

layout(std430, binding = 0) buffer IndirectDrawBuffer {
    DrawElementsIndirectCommand commands[];
};

layout(std430, binding = 1) buffer DrawCountBuffer {
    uint drawCount;
};

Visibility Testing via Compute Shaders

The visibility evaluation begins by dispatching a compute shader where each thread or thread group is responsible for a single bounding box (AABB or oriented bounding box) representing a mesh or instance:

  1. Transforming the Bounding Box: The compute shader transforms the eight corners of the mesh's bounding box into clip space.
  2. Frustum Culling: An initial check determines if all points lie outside any clip plane. If outside, the object is culled immediately.
  3. Screen-Space Footprint: The bounding box is projected onto screen space to calculate a bounding rectangle and determine the appropriate mip level of a Hierarchical Z-Buffer (Hi-Z).
  4. Depth Comparison: The maximum depth value of the bounding box is compared against the minimum depth fetched from the Hi-Z pyramid. If the bounding box depth is further than the stored depth, the object is occluded.

Writing to the Indirect Buffer

Once a mesh or instance is determined to be visible, the compute shader must record this result into the indirect buffer. This can be achieved through two primary patterns depending on whether you are compacting instances or individual draw commands.

Pattern 1: Per-Command Instance Compacting

When drawing instances of the same mesh (e.g., foliage or crowds), the indirect draw command already exists in the buffer, and the compute shader simply increments the instanceCount field:

bool visible = performOcclusionTest(instanceID);

if (visible) {
    // Atomically increment the instance count for this draw command
    uint instanceIndex = atomicAdd(commands[meshID].instanceCount, 1u);
    
    // Store visible instance data in an instance-data buffer
    visibleInstanceIDs[meshID][instanceIndex] = instanceID;
}

Prior to dispatching the culling compute shader, the CPU or a separate initialization compute shader resets instanceCount to 0.

Pattern 2: Multi-Draw Command Compacting

When culling heterogeneous objects using glMultiDrawElementsIndirectCount (or standard multi-draw), each visible object receives its own draw command slot in the output buffer. The compute shader appends fully populated command structures using an atomic counter:

bool visible = performOcclusionTest(objectID);

if (visible) {
    // Reserve an index in the indirect buffer
    uint cmdIndex = atomicAdd(drawCount, 1u);

    // Populate the indirect command
    commands[cmdIndex].count = meshData[objectID].indexCount;
    commands[cmdIndex].instanceCount = 1u;
    commands[cmdIndex].firstIndex = meshData[objectID].firstIndex;
    commands[cmdIndex].baseVertex = meshData[objectID].baseVertex;
    commands[cmdIndex].baseInstance = objectID;
}

The drawCount buffer corresponds directly to the parameter buffer required by glMultiDrawElementsIndirectCountARB, enabling the GPU to determine both the commands and the total number of commands to execute.

Synchronization and Execution

Because compute shaders write to the buffer asynchronously, memory barriers must be injected before invoking the render pass on the CPU side:

// Dispatch compute culling
glDispatchCompute(numWorkGroups, 1, 1);

// Ensure writes to SSBO and Indirect buffers are complete
glMemoryBarrier(GL_COMMAND_BARRIER_BIT | GL_SHADER_STORAGE_BARRIER_BIT);

// Execute the indirect draw
glBindBuffer(GL_DRAW_INDIRECT_BUFFER, indirectBufferHandle);
glBindBuffer(GL_PARAMETER_BUFFER, countBufferHandle);

glMultiDrawElementsIndirectCount(
    GL_TRIANGLES,
    GL_UNSIGNED_INT,
    (const void*)0,      // Indirect buffer offset
    0,                   // Count buffer offset
    maxDrawCommands,     // Upper bound limit
    sizeof(DrawElementsIndirectCommand)
);

Using GL_COMMAND_BARRIER_BIT guarantees that the indirect dispatch hardware waits until the compute shader has finished writing visibility draw counts and command structures before the draw calls begin processing.