Improving GLSL Pixel Fill Rate with Early Fragment Tests

In fill-rate bound GLSL rendering pipelines, performance degrades when the GPU spends valuable compute cycles and memory bandwidth processing pixels that are ultimately hidden by other geometry. Early fragment testing optimizes this pipeline by moving depth and stencil operations ahead of fragment shader execution. This article explains how early fragment test failures discard occluded fragments before costly shading occurs, dramatically reducing redundant computations, conserving bandwidth, and maximizing effective pixel fill rate.

The Fill-Rate Bottleneck

A render pass becomes fill-rate bound when the GPU cannot write fragments to the framebuffer fast enough to match the target frame rate. This bottleneck typically arises from high screen resolutions, complex fragment shaders (heavy texture sampling, dynamic branching, or multi-light evaluations), and excessive overdraw. Overdraw occurs when a single pixel on the screen is shaded multiple times per frame by different overlapping primitives, with only the closest primitive surviving to the final image.

Traditional vs. Early Fragment Testing

In the standard OpenGL/GLSL pipeline, fragment operations follow this sequence:

  1. Primitive rasterization generates fragment candidates.
  2. The fragment shader executes, computing color, material properties, and potentially a custom depth value.
  3. Per-fragment operations (depth and stencil tests) evaluate whether the fragment is visible.
  4. Surviving fragments are written to the framebuffer; failing fragments are discarded.

When testing occurs after shading, the GPU executes the entire fragment shader even if the fragment is immediately discarded by a failed depth test.

Early fragment testing reorders this sequence, executing the depth and stencil tests before the fragment shader runs. If a fragment fails the depth comparison against the existing depth buffer (for example, if an object sits behind already-drawn opaque geometry), the pipeline drops the fragment immediately.

How Test Failures Boost Pixel Fill Rate

When an early fragment test fails, it provides three direct performance benefits that relieve fill-rate bottlenecks:

  • Elimination of Shader ALU Overhead: The fragment shader is completely bypassed for failed fragments. Complex mathematical instructions, procedural noise generation, and lighting loops are skipped, freeing shader cores to process visible pixels.
  • Texture Bandwidth Conservation: Bypassing the shader prevents texture fetches. This prevents texture cache thrashing and preserves memory bandwidth, which is often the primary limiter in fill-rate constrained applications.
  • Avoided Framebuffer Contention: Because rejected fragments do not proceed to write operations, the pipeline reduces write traffic and atomic blend contention at the memory controller level.

While the GPU's theoretical maximum pixel throughput (the raw hardware fill rate) remains unchanged, the effective fill rate increases substantially. Processing power is dedicated exclusively to visible surfaces rather than wasted on discarded work.

Maximizing Early Test Failures

To leverage early fragment tests effectively in GLSL passes, developers must align both application-level sorting and shader design:

  1. Front-to-Back Rendering Order: Sort opaque geometry from nearest to farthest relative to the camera. The closest objects write their depth values to the buffer first, causing subsequent overlapping fragments from farther objects to fail their early depth tests.

  2. Explicit Shader Qualifiers: In GLSL 4.2 and later, you can enforce early tests using the input layout qualifier:

    layout(early_fragment_tests) in;

  3. Avoid Disqualifying Shader Operations: Modern GPU drivers attempt to enable early depth tests automatically, but certain shader operations force the GPU to fall back to late fragment testing. Modifying gl_FragDepth or using the discard keyword alters fragment visibility or depth dynamically, preventing the hardware from determining visibility before the shader completes. Keeping fragment depth writes disabled ensures early testing stays active.