SMAA Blending Weight Calculation in GLSL

Subpixel Morphological Anti-Aliasing (SMAA) evaluates blending weights in a dedicated second pass that determines how adjacent pixels blend together along detected edges. By analyzing continuous edge lines, finding their endpoints, and classifying the underlying geometric patterns, the GLSL implementation converts discrete pixel steps into smooth, subpixel-accurate coverage weights using precomputed lookup textures.

Input and Coordinate Setup

The blending weight pass operates on the output of the edge detection pass, which stores horizontal edges in the red channel and vertical edges in the green channel. In the GLSL fragment shader (SMAABlendingWeightCalculationPS), the pipeline evaluates horizontal edges first to produce top/bottom blending weights, followed by vertical edges to produce left/right blending weights.

The shader calculates normalized texture coordinates and takes advantage of bilinear filtering offsets to reduce the total number of texture fetches required to sample the edge buffer.

Edge Distance Searching

To determine the length of an edge, the shader searches linearly in both directions from the current pixel until it identifies a discontinuity or reaches a predefined maximum search step limit.

In GLSL, this is handled by helper functions such as SMAASearchXLeft, SMAASearchXRight, SMAASearchYUp, and SMAASearchYDown. These functions do not sample every individual texel linearly; instead, they sample a precomputed 2D search texture (searchTex) using bilinear interpolation. The search texture encodes step distances, allowing the shader to skip multiple pixels per sample and quickly return the distance offsets (\(d_1\) and \(d_2\)) to the edge endpoints.

Pattern Classification

Once the endpoints of the edge segment are located, the shader samples the edge texture orthogonally at these terminal positions. This step checks for "crossing edges"—perpendicular edge segments that define the structural shape of the boundary:

  • L-shapes: The edge turns in one direction at one endpoint.
  • Z-shapes: The edge turns in opposite directions at both endpoints.
  • U-shapes: The edge turns in the same direction at both endpoints.

The values of these crossing edges, combined with the relative distances to the endpoints, uniquely define the geometric trapezoid or triangle representing the virtual continuous edge passing through the pixel.

Area Texture Lookup

Directly calculating the analytical coverage of arbitrary trapezoids in real time is computationally expensive. SMAA offloads this math to a precomputed 2D lookup table packed into a texture (areaTex).

The GLSL shader maps the extracted parameters directly into texture coordinates for areaTex:

  1. The distances to the left/top (\(d_1\)) and right/bottom (\(d_2\)) endpoints are scaled and mapped to coordinate axes.
  2. The crossing edge configurations at the endpoints are converted into discrete coordinate offsets to select the appropriate sub-table within the area texture.
  3. A bilinear sample of areaTex retrieves the precise coverage area value, representing the fraction of the neighboring pixel that should be blended into the current pixel.

Output Packing

The shader computes coverage factors for both orthogonal directions. For a horizontal edge, it outputs blending weights for the top and bottom neighbors; for a vertical edge, it outputs blending weights for the left and right neighbors. These four floating-point scalar weights are packed into a single RGBA target:

  • Red: Left blending weight
  • Green: Right blending weight
  • Blue: Top blending weight
  • Alpha: Bottom blending weight

This RGBA texture is directly bound as the input for the third and final pass of SMAA, where neighborhood blending is executed using linear filtering based on these stored weights.