How Does an OpenGL Fragment Shader Determine Pixel Color?

The OpenGL fragment shader determines the final color of a pixel by evaluating mathematical models, interpolated vertex attributes, texture data, and light sources for every potential screen pixel, or "fragment." While the shader calculates an initial RGBA color value per fragment, that value undergoes several mandatory per-fragment operations—such as depth testing, stencil testing, and alpha blending—before the graphics processing unit (GPU) officially commits the final pixel to the framebuffer.

The Role of the Rasterizer and Interpolation

Before a fragment shader executes, the OpenGL graphics pipeline transforms 3D geometry into primitive shapes like triangles during the vertex and geometry stages. Once primitives are positioned in screen space, the rasterizer breaks each primitive down into discrete fragments. A fragment represents all the data necessary to update a single pixel in the framebuffer.

The rasterizer performs perspective-correct interpolation on all data passed out of the vertex shader, such as UV texture coordinates, surface normals, and tangent vectors. If two vertices of a line have red and blue values respectively, the rasterizer computes a smooth gradient of values for every fragment along that line. The fragment shader receives these interpolated inputs as its starting baseline for color calculations.

Sampling Textures and Material Properties

Most 3D models rely on texture mapping to define intricate surface details without extra geometry. Within the fragment shader, developers declare texture samplers (sampler2D, samplerCube) that reference image data stored in GPU memory.

Using the interpolated UV coordinates, the shader samples color values from one or more textures:

• Diffuse/Albedo Maps: Provide the base color and pattern of the surface.
• Specular/Roughness Maps: Define how shiny or rough distinct areas of the surface are.
• Normal Maps: Alter the surface normal vectors per fragment to simulate bumps, grooves, and complex physical relief under lighting.

The fragment shader can sample multiple textures simultaneously and blend them mathematically based on masks, vertex weights, or procedural functions.

Lighting and Material Calculations

Raw texture color alone produces flat, lifeless visuals. To create realism, the fragment shader computes lighting models by evaluating vectors between the fragment position, light sources, surface normals, and the camera viewpoint.

Modern engines implement models ranging from basic empirical calculations to physically based rendering (PBR):

• Phong and Blinn-Phong Models: Calculate color by combining three components: ambient (uniform base illumination), diffuse (light scattered evenly across rough surfaces), and specular (bright highlights reflecting directly into the camera).
• Physically Based Rendering (PBR): Uses equations like the Cook-Torrance microfacet specular shading model alongside bidirectional reflectance distribution functions (BRDF). These calculations account for energy conservation, surface roughness, and metallic reflectance to determine exact light interactions.

The final color output typically sums the contributions of all directional, point, and spotlights affecting the fragment, modulated by shadows computed via depth maps.

Post-Lighting Effects and Discard Logic

Inside the shader code, developers can modify or reject colors before outputting a result:

• Alpha Testing and Discarding: A fragment shader can invoke the discard keyword. If an object uses transparent cutouts (like foliage or chain-link fences), the shader evaluates the texture's alpha value. If it falls below a threshold, discard terminates execution for that fragment, ensuring no color or depth is written.
• Color Grading and Tone Mapping: For High Dynamic Range (HDR) rendering, colors often exceed the standard 0.0 to 1.0 range. The shader applies tone mapping curves (such as Reinhard or ACES) and gamma correction to map HDR values into standard displayable color space.

At the end of main execution, the shader assigns its computed values to an output variable:

out vec4 FragColor;

void main()
{
    FragColor = vec4(finalColor, alpha);
}

Per-Sample Operations and the Framebuffer

Writing to FragColor does not immediately update the monitor. The fragment must survive a sequence of fixed-function GPU tests:

1. Scissor and Stencil Tests: Verify whether the fragment falls inside specific render boundaries or passes masking conditions defined in the stencil buffer.
2. Depth Test: Compares the fragment's Z-depth against the current value in the depth buffer. If an existing object is closer to the camera, the new fragment is rejected.
3. Blending: If the fragment passes all tests and has an alpha value less than 1.0 (with blending enabled), OpenGL blends the fragment's color with the existing pixel color in the framebuffer using blend equations like standard alpha compositing.
4. Dithering and Logical Operations: Optional final passes apply subtle noise patterns or bitwise logic to reduce color banding on displays with limited color depth.

Once these stages complete, the GPU writes the resulting RGBA value directly to the active framebuffer, establishing the visible pixel displayed on screen.