How GLSL Discard Affects Early Depth Testing

Using the discard keyword in a GLSL fragment shader disrupts the GPU's ability to perform early depth and stencil testing (commonly referred to as Early-Z), causing the pipeline to fall back to late fragment testing. Because the GPU cannot predict whether a fragment will survive execution until the shader finishes, it must defer depth and stencil updates to avoid corrupting buffer data with discarded pixels. This overview explains the mechanics behind this behavior, its performance ramifications, and strategies to mitigate the resulting overhead.

The Purpose of Early Depth and Stencil Testing

In a standard rendering pipeline, fragment shading occurs before depth and stencil testing. However, modern GPUs implement an optimization called Early-Z. When enabled, the hardware evaluates depth and stencil values before invoking the fragment shader.

If a fragment is determined to be occluded by previously rendered geometry, the GPU discards it immediately. This skips execution of the fragment shader entirely, saving significant processing power and memory bandwidth on complex lighting, texturing, and arithmetic calculations.

Why discard Breaks Early Depth Testing

The discard keyword conditionally terminates a fragment's execution, preventing it from writing color, depth, or stencil data to the framebuffer. This introduces an architectural conflict:

  1. Depth Buffer Corruption: If the GPU performed early depth writes, a fragment that updates the depth buffer early and subsequently executes a discard would leave behind an invalid depth value. Any geometry rendered behind this discarded pixel would fail future depth tests, creating visual occlusion artifacts (invisible geometry).
  2. Pipeline Fallback (Late-Z): To preserve rendering correctness, the GPU driver automatically disables early depth writes whenever a shader contains a discard statement. On most architectures, this causes both the depth test and depth write to shift to the end of the pipeline (Late-Z), running after the fragment shader finishes.
  3. Partial Early-Z (Hierarchical-Z / Read-Only): Some modern GPUs can still perform an early depth read (testing without writing) to reject fragments occluded by existing geometry. However, if the fragment passes the depth test, the GPU must still execute the fragment shader to see if it discards before finally committing the depth write at the end of the stage.

Impact on the Stencil Buffer

The stencil buffer behaves similarly to depth testing. When a fragment shader contains discard, any stencil operations—such as incrementing, replacing, or zeroing stencil values—cannot be committed prior to shader completion. If stencil operations depend on fragment survival, the stencil test must be postponed to the late testing stage, compounding pipeline stalls and disabling hardware-level stencil culling optimizations.

Performance Consequences

Disabling early testing through discard leads to severe performance penalties, particularly regarding fill rate and overdraw:

  • Overdraw Overhead: In scenes with heavy overdraw (such as dense foliage, hair, or overlapping transparent cutouts), the GPU must execute full fragment shaders for fragments that may end up discarded or later overwritten.
  • Instruction Stalls: Branch divergence occurs when adjacent threads within a GPU warp or wavefront take different paths—some discarding while others continue—reducing overall SIMD execution efficiency.

Mitigations and Best Practices

To minimize the impact of discard on early depth testing, several techniques can be utilized:

  • Depth Pre-Pass (Z-Prepass): Render the scene's geometry to the depth buffer first using a minimal shader, then render the color pass with depth writes disabled and depth testing set to GL_EQUAL or GL_LEQUAL. This guarantees all occluded fragments are culled before the main color shader runs.

  • Front-to-Back Sorting: When rendering objects utilizing discard (such as alpha-tested foliage), sort draw calls strictly from front to back to maximize any partial early depth rejection capabilities available on the hardware.

  • Explicit Early Fragment Tests: In OpenGL 4.2 and higher, shaders can force early testing using the layout qualifier:

    layout(early_fragment_tests) in;

    This forces depth and stencil tests and writes to happen before shader execution. However, if discard is used inside this shader, the depth and stencil buffers will retain the fragment's values even if the fragment is discarded, which alters visual results unless the application explicitly accounts for it.

  • Alpha-to-Coverage: For multi-sampled anti-aliasing (MSAA), alpha-to-coverage converts fragment alpha into a sample mask. This enables perforated transparency effects without invoking discard, keeping early depth optimizations intact.