Temporal Reprojection and Ghosting in OpenGL TAA

Temporal reprojection is a core rendering technique within Temporal Anti-Aliasing (TAA) that aligns historical frame data with the current frame using per-pixel motion vectors. In an OpenGL rendering pipeline, TAA accumulates sub-pixel jittered samples over time to produce a smooth, high-quality image without the performance cost of traditional multi-sampling. However, movement of objects or the camera can cause past color data to misalign with current geometry, creating smear-like visual artifacts known as ghosting. Temporal reprojection resolves this by accurately mapping where current pixels existed in the prior frame and discarding invalid historical data.

The Role of Temporal Reprojection in TAA

TAA jittered camera matrices produce a different sub-pixel offset each frame. If the scene is completely static, accumulating these frames into an ongoing accumulation buffer resolves edges effectively. Once the camera or objects move, naive accumulation blends pixels representing entirely different surfaces, resulting in severe motion blur and ghosting.

Temporal reprojection remedies this by transforming the previous frame's color buffer into the current frame's screen space. To accomplish this, the rendering pipeline generates a motion vector (or velocity) buffer.

In OpenGL, this is typically handled in a fragment shader during the G-buffer pass (in deferred rendering) or via a dedicated velocity pre-pass:

  1. Current Screen Position: Calculated from the current clip space coordinate \(P_{current} = M_{proj} \times M_{view} \times M_{model} \times P_{local}\).
  2. Previous Screen Position: Calculated using the previous frame’s matrices \(P_{prev} = M_{prev\_proj} \times M_{prev\_view} \times M_{prev\_model} \times P_{local}\).
  3. Motion Vector Calculation: Both positions are converted to normalized device coordinates (NDC) via perspective division, and their difference \((\text{NDC}_{current} - \text{NDC}_{prev})\) represents the screen-space velocity stored into an RG16F or RG32F frame-buffer texture.

During the TAA resolve pass, the fragment shader samples this velocity texture to offset the texture coordinates used to sample the history buffer:

\[\text{HistoryUV} = \text{CurrentUV} - \text{Velocity}\]

How Reprojection Prevents Ghosting

Simply sampling the reprojected location is insufficient to eliminate ghosting entirely. Disocclusion—where an object moves and reveals a surface that was hidden in the previous frame—means the historical buffer contains color information for a surface that did not exist at those coordinates previously. Without mitigation, this past color bleeds into the new surface.

To prevent ghosting under these conditions, modern OpenGL TAA pipelines employ three primary techniques:

1. Color Clipping and Neighborhood Clamping

The most robust defense against ghosting is neighborhood bounding. In the TAA resolve shader, the current frame’s color is sampled alongside its immediate neighbors (typically a \(3 \times 3\) kernel). These samples establish an axis-aligned bounding box (AABB) in color space:

  • The minimum and maximum color values (or luminance-chrominance channels using spaces like YCoCg) within the \(3 \times 3\) grid define the valid color envelope.
  • The reprojected historical sample is clamped or clipped against this bounding box before blending.
  • If a disocclusion occurs, the reprojected color falls far outside the current neighborhood’s color distribution, and the clamp forcefully pulls it within valid range, immediately suppressing the ghost artifact.

2. Depth and Normal Validation

Reprojected coordinates can also be validated by comparing geometric characteristics. By storing a linear depth buffer and a world-space normal buffer for both the current and previous frames:

  • The current pixel's depth is compared against the depth sampled at the reprojected coordinate in the previous depth buffer.
  • If the difference exceeds a small threshold, or if the surface normals deviate significantly, the pixel is marked as disoccluded.
  • The historical sample's contribution weight is reduced or discarded completely, forcing the pipeline to rely solely on the current frame's data until history stabilizes.

3. Responsive Blending Weights

TAA combines history and current samples using an exponential moving average:

\[\text{Color}_{final} = \text{mix}(\text{Color}_{history}, \text{Color}_{current}, \alpha)\]

Under stable conditions, \(\alpha\) is kept low (typically between \(0.05\) and \(0.10\)) to preserve anti-aliasing quality across time. When motion vectors or neighborhood clipping indicate rapid movement or large discrepancies, \(\alpha\) is dynamically increased toward \(1.0\). This rapidly flushes outdated temporal history and replaces it with newly rendered information, keeping the resulting image sharp and free of trail artifacts.