glTF EXT_lights_image_based Environment Lighting

The EXT_lights_image_based extension provides a standardized way to define high-dynamic-range (HDR) environment lighting directly within glTF 2.0 assets. By decoupling lighting from flat ambient colors or localized punctual lights, this extension enables realistic Image-Based Lighting (IBL) for physically based rendering (PBR). This article breaks down how the extension structures environment data, processes diffuse irradiance and specular reflections, and maps these components to 3D scenes.

Top-Level Extension Architecture

In a standard glTF asset, lighting typically relies on punctual sources (such as directional, point, or spot lights defined by KHR_lights_punctual). The EXT_lights_image_based extension introduces an array of image-based light objects at the root level of the glTF document under extensions.EXT_lights_image_based.lights.

Scenes or nodes then reference these environment lights by index. When applied to a scene, the light acts as a global skybox and distant ambient illuminator. When applied to a specific node, the environment light moves and rotates according to that node's spatial transform hierarchy, allowing dynamic or localized environmental shifts.

Diffuse Irradiance via Spherical Harmonics

Rather than requiring an expensive runtime blur of an environment map or storing a large irradiance cubemap, EXT_lights_image_based defines diffuse environmental light using spherical harmonics.

The extension specifies a set of coefficients—typically 9 RGB vector coefficients representing 3rd-order (L2) spherical harmonics. These coefficients mathematically encode low-frequency color and intensity information collected from the entire surrounding environment. At render time, shaders can evaluate these harmonics using only the surface normal vector, providing mathematically smooth, direction-dependent diffuse ambient lighting with negligible memory overhead and fast GPU execution.

Specular Reflections via Prefiltered Environment Maps

To accurately simulate glossy and mirror-like reflections on metallic or dielectric surfaces, the extension uses prefiltered radiance environment maps.

  • Cubemap Structure: The specular data is stored as a cubemap texture containing six square faces per level of detail.
  • Roughness Mipmapping: The cubemap utilizes a mipmap chain where each descending mip level corresponds to an increasing perceptual roughness value. Smooth surfaces (roughness near 0) sample the highest-resolution, sharpest mip level, while rough surfaces sample progressively lower, blurrier mip levels.
  • Split-Sum Approximation: This data is tailored for the split-sum approximation commonly used in the glTF PBR shading model. When combined with a standard 2D Look-Up Table (BRDF LUT) in the engine, the prefiltered map produces accurate microfacet specular reflections without requiring real-time ray tracing.

Intensity and Spatial Orientation

The extension allows digital content creation tools and engines to align the environment accurately with the virtual world:

  • Intensity: An intensity scalar scales the linear brightness of both the diffuse and specular contributions, adjusting exposure without modifying the underlying HDR texture data.
  • Rotation: A rotation property (defined as a unit quaternion) sets the orientation of the environment light relative to the coordinate space of the scene or parent node. This allows the skybox and its corresponding reflections to be aligned with custom directional elements, such as a dominant sun position.

Runtime Integration with the PBR Pipeline

When a client engine renders a glTF model using EXT_lights_image_based, it combines the image data with standard glTF PBR material inputs:

  1. Diffuse Term: The surface normal looks up the diffuse irradiance from the spherical harmonics, which is multiplied by the material’s base color and balanced against the metallic value.
  2. Specular Term: The reflection vector and surface roughness determine which mip level of the prefiltered cubemap to sample. This result is scaled by the Fresnel reflectance curve and the material's roughness-metallic properties.
  3. Occlusion: Standard material ambient occlusion maps mask both the diffuse and specular results, preventing ambient light leaks in recessed geometries.