Fixing Thin Geometry Ghosting in Temporal Upscaling

Temporal super-resolution (TSR) techniques reconstruct high-resolution frames using lower-resolution inputs and data accumulated over time, but thin geometries like wires, chain-link fences, and foliage frequently suffer from ghosting due to sub-pixel discontinuities and velocity discrepancies. Engineers overcome this artifact by refining velocity vector retrieval via depth dilation, utilizing reactive masks to dynamically adjust temporal accumulation, applying variance-based color clamping, and deploying disocclusion heuristics. Together, these techniques ensure that high-frequency, sub-pixel features retain sharpness across frames without leaving distracting trails behind fast-moving objects.

The Problem with Thin Geometry

Temporal super-resolution algorithms rely on motion vectors to map current-frame pixels to their positions in the previous history buffer. When an object is thinner than a pixel or moves quickly across a high-contrast background, several issues arise:

  • Sub-pixel aliasing: The thin object might be sampled in one frame and missed entirely in the next due to camera jittering.
  • Motion vector bleeding: The motion vector calculated for a pixel containing a thin edge often captures the background velocity rather than the foreground velocity, or vice versa.
  • Rapid occlusion and disocclusion: The background behind a thin wire reveals and hides itself continuously, invalidating the historical color data at a high frequency.

When stale history data blends with the current frame using incorrect motion vectors, the engine produces ghosting.

Depth-Dilated Motion Vectors

Standard motion vector buffers often fail at edges because linear texture filtering or rasterization averages foreground and background movement. To ensure thin geometry moves with its own velocity rather than the background's:

  1. 3x3 Depth Search: For a given pixel, the shader inspects a \(3\times3\) neighborhood in the depth buffer.
  2. Closest Depth Selection: The algorithm identifies the sample closest to the camera (the smallest depth value in reverse-Z setups).
  3. Velocity Snapping: The motion vector from the nearest depth sample is assigned to the current pixel.

Because thin features like wires almost always sit in front of distant backgrounds, this "dilation" ensures the upscaler projects the geometry using foreground motion, preventing background vectors from smearing the thin line.

Reactive Masks

Modern upscaling frameworks, including AMD FidelityFX Super Resolution (FSR) and Unreal Engine's TSR, utilize dedicated reactive masks to bypass temporal accumulation on problematic surfaces.

  • Generation: The reactive mask is generated during rendering by comparing the pre-pass and post-pass scene data or explicitly tagging materials (such as alpha-tested foliage, particle effects, or hair).
  • Function: The mask value (from 0.0 to 1.0) dictates the temporal blend factor. A value near 1.0 forces the upscaler to rely heavily on the current frame's spatial samples rather than the temporal history.
  • Application to Thin Features: Forcing a higher spatial weight on thin, translucent, or sub-pixel elements eliminates the accumulation of stale history, completely removing ghost trails at the expense of slight temporal instability, which is far less noticeable than trailing smudges.

Advanced History Clamping and Variance Clipping

To prevent invalid historical colors from bleeding into the current frame, upscalers bound the history color using the neighborhood of the current frame.

  • Color Space Transformation: Pixel values are typically transformed into a perceptual, luma-chroma color space such as YCoCg, which separates brightness from color data.
  • Axis-Aligned Bounding Box (AABB): The shader computes the minimum and maximum color values within a \(3\times3\) neighborhood around the pixel in the current frame.
  • Variance Clipping: Instead of a strict minimum/maximum clamp—which can be too loose or cause flickers—modern engines compute the statistical mean (\(\mu\)) and standard deviation (\(\sigma\)) of the neighborhood. The valid color range is clamped to \([\mu - \beta\sigma, \mu + \beta\sigma]\), where \(\beta\) is a tuning constant.
  • Thin Geometry Adjustments: If the neighborhood variance is large (indicating a high-contrast thin edge), \(\beta\) can be dynamically tightened to reject history more aggressively, instantly clipping out ghost trails.

Sub-Pixel Jitter and Edge-Preserving Reconstruction Filters

Temporal accumulation requires sub-pixel jitter patterns (such as Halton sequences) to resolve details higher than the native rendering resolution. When reconstructing thin edges:

  • Sharper Kernel Weights: Instead of broad Gaussian filters that blur sub-pixel details into adjacent pixels, developers employ sharper reconstruction kernels, such as Lanczos, Catmull-Rom, or Mitchell-Netravali.
  • Edge Direction Weighting: Reconstruction passes analyze local gradient directions to align filtering along the edge rather than across it. This prevents the thin geometry from expanding across multiple pixels, preserving fine contours and allowing temporal clamps to correctly isolate the feature.