Smith GGX Geometry Shadowing in glTF Shading

This article explains the critical role of the Smith GGX geometry shadowing function within the glTF 2.0 standard. It covers how the function models microfacet occlusion, integrates into the Cook-Torrance bidirectional reflectance distribution function (BRDF), and guarantees physically accurate, energy-conserving specular reflections across real-time 3D graphics engines.

The Microfacet Model and the Cook-Torrance BRDF

In glTF’s standard metallic-roughness Physically Based Rendering (PBR) workflow, light interaction with a surface is computed using the Cook-Torrance microfacet specular BRDF. This model treats a microscopic surface as a collection of perfectly reflective microfacets oriented randomly according to surface roughness. The equation is defined as:

\[f_r = \frac{D \cdot F \cdot G}{4(\mathbf{n} \cdot \mathbf{v})(\mathbf{n} \cdot \mathbf{l})}\]

Where:

  • \(D\) is the microfacet Normal Distribution Function (NDF), represented in glTF by the GGX/Trowbridge-Reitz distribution.
  • \(F\) is the Fresnel term, which models reflectance based on the viewing angle using the Schlick approximation.
  • \(G\) is the Geometry function, which accounts for the occlusion of microfacets.
  • \(\mathbf{n} \cdot \mathbf{v}\) and \(\mathbf{n} \cdot \mathbf{l}\) are the dot products of the surface normal with the view and light directions, respectively.

The Role of the Geometry Function (\(G\))

Rough surfaces feature microscopic peaks and valleys. When light strikes these irregularities, two physical phenomena occur:

  1. Shadowing: Incoming light is blocked by adjacent microfacets before reaching a reflective surface.
  2. Masking: Reflected light is blocked by adjacent microfacets before reaching the camera or observer.

Without a geometry function to reduce light output at shallow angles, a rendered surface reflects more energy than it receives. This produces unnatural edge highlights and breaks the law of conservation of energy. The geometry function models this loss of light, attenuating specular brightness based on roughness and angle.

The Smith GGX Formulation in glTF

The glTF specification defines the use of the Smith method adapted specifically for the GGX distribution. The Smith formulation assumes microfacet heights and slopes are statistically independent, calculating the shadowing and masking components separately:

\[G(\mathbf{l}, \mathbf{v}, \mathbf{h}) = G_1(\mathbf{l}) \cdot G_1(\mathbf{v})\]

Height-Correlated Smith Model

Rather than calculating independent probabilities, modern glTF implementations use the height-correlated Smith GGX formulation. This accounts for the statistical correlation between the incoming light ray and the outgoing viewing ray: microfacets positioned higher up on the surface are simultaneously less likely to be shadowed and less likely to be masked.

The height-correlated function is formulated as:

\[G(\mathbf{l}, \mathbf{v}) = \frac{2(\mathbf{n} \cdot \mathbf{v})(\mathbf{n} \cdot \mathbf{l})}{(\mathbf{n} \cdot \mathbf{l})\sqrt{\alpha^2 + (1 - \alpha^2)(\mathbf{n} \cdot \mathbf{v})^2} + (\mathbf{n} \cdot \mathbf{v})\sqrt{\alpha^2 + (1 - \alpha^2)(\mathbf{n} \cdot \mathbf{l})^2}}\]

Where \(\alpha\) represents the roughness parameter mapped quadratically from the glTF roughness input (\(\alpha = \text{roughness}^2\)).

Optimization via the Visibility Function (\(V\))

In real-time rendering, glTF shaders combine the geometry function with the Cook-Torrance denominator into a single term known as the Visibility function (\(V\)):

\[V = \frac{G}{4(\mathbf{n} \cdot \mathbf{v})(\mathbf{n} \cdot \mathbf{l})}\]

By combining these calculations, common factors cancel out, producing an optimized equation:

\[V = \frac{0.5}{(\mathbf{n} \cdot \mathbf{l})\sqrt{\alpha^2 + (1 - \alpha^2)(\mathbf{n} \cdot \mathbf{v})^2} + (\mathbf{n} \cdot \mathbf{v})\sqrt{\alpha^2 + (1 - \alpha^2)(\mathbf{n} \cdot \mathbf{l})^2}}\]

This mathematical simplification allows real-time engines to compute accurate specular reflections using minimal GPU instructions.

Visual and Technical Importance

The Smith GGX geometry function provides several key benefits in the glTF pipeline:

  • Elimination of Grazing-Angle Artifacts: It suppresses glowing white silhouettes that occur on rough surfaces viewed at glancing angles.
  • Consistent Material Appearance: Specifying the exact Smith GGX formulation ensures that a glTF asset looks identical when rendered across different engines, such as Three.js, Babylon.js, Blender, or Unreal Engine.
  • Roughness Scaling: At a roughness of zero (\(\alpha \to 0\)), the function evaluates cleanly to mirror reflection behavior, smoothly transitioning to matte, diffuse-like scattering as roughness approaches one.