GLSL Depth Peeling for Order-Independent Transparency

Order-independent transparency (OIT) addresses the visual artifacts caused by rendering overlapping transparent surfaces without prior depth sorting. The classic multi-pass depth peeling algorithm, originally introduced by Cass Everitt, solves this by iteratively extracting and compositing geometric layers from front to back (or back to front) using the GPU depth buffer. In GLSL, this algorithm operates across multiple render passes by comparing the current fragment's depth against the depth recorded in the previous pass, peeling away the nearest surfaces layer by layer until the scene is fully composited.

The Depth Peeling Algorithm

Traditional rendering requires transparent primitives to be sorted back-to-front on the CPU to blend correctly using the standard alpha compositing operator. Depth peeling bypasses geometric sorting entirely by using depth-buffer testing inside fragment shaders to segment the geometry into discrete depth layers.

The multi-pass algorithm relies on two depth textures that alternate roles (ping-ponging) each pass:

  1. Previous Depth Texture: A read-only texture containing the depth values of the layer peeled in the previous iteration.
  2. Current Depth Texture: A write-enabled depth buffer bound to the active Framebuffer Object (FBO) to capture the next nearest surface.

GLSL Implementation Workflow

The implementation consists of an initialization pass followed by an iterative peeling loop.

1. First Pass (Nearest Layer)

Render the transparent geometry using standard depth testing (GL_LESS). This writes the depth and color of the absolute closest transparent surface into the first layer target. No peeling logic is needed in the shader for this initial step.

2. Subsequent Peeling Passes

For each subsequent layer, bind the depth buffer from the previous pass as a texture sampler. In the GLSL fragment shader, fetch the previous depth at the current screen coordinate and discard any fragment that is closer than or equal to that depth:

#version 330 core

uniform sampler2D uPrevDepthTex;
uniform vec4 uColor;

out vec4 fragColor;

void main()
{
    // Retrieve depth from the previous pass
    ivec2 texCoord = ivec2(gl_FragCoord.xy);
    float prevDepth = texelFetch(uPrevDepthTex, texCoord, 0).r;

    // Discard fragments already captured in earlier passes
    if (gl_FragCoord.z <= prevDepth) {
        discard;
    }

    // Output fragment color for the current layer
    fragColor = uColor;
}

With hardware depth testing still active (GL_LESS), the fragments that survive the discard check compete normally. The depth test naturally resolves to the next closest fragment behind the previously rendered layer.

Compositing the Layers

Depth peeling typically uses front-to-back compositing into an accumulation color buffer. Because fragments are processed from front to back, blending requires the under-operator rather than standard alpha blending:

  • Destination Color: \(C_{dst} = C_{dst} + (1 - A_{dst}) \times (A_{src} \times C_{src})\)
  • Destination Alpha: \(A_{dst} = A_{dst} + (1 - A_{dst}) \times A_{src}\)

In OpenGL, this can be configured using:

glEnable(GL_BLEND);
glBlendEquation(GL_FUNC_ADD);
glBlendFuncSeparate(GL_ONE_MINUS_DST_ALPHA, GL_ONE, GL_ONE_MINUS_DST_ALPHA, GL_ONE);

Termination and Optimization

The loop repeats until a fixed maximum number of layers is reached or until no fragments are rendered. Dynamic termination is achieved using hardware occlusion queries (GL_SAMPLES_PASSED). When a pass renders zero samples, all visible layers have been peeled, and execution breaks.

To reduce the number of passes, the Dual Depth Peeling variant can be implemented. It uses two depth attachments with MAX and MIN blending to peel the front-most and back-most layers simultaneously in a single pass, cutting the required draw iterations in half.