Deep Shadow Maps for Hair Transmittance in GLSL

Deep shadow maps solve the challenge of rendering self-shadowing in dense hair volumes by replacing binary depth tests with continuous, fractional light transmittance functions. While standard shadow mapping records only the closest occluder, deep shadow maps store a transmittance curve along each light ray, calculating how much light survives after passing through multiple semi-transparent hair strands. In OpenGL Shading Language (GLSL), this is achieved by capturing depth and absorption values per strand, compressing these points into a piecewise representation, and sampling the resulting attenuation function during the final lighting pass.

The Limitations of Traditional Shadow Maps

Standard shadow maps record a single floating-point depth value \(z\) representing the first surface hit from the light’s perspective. Hair geometry, however, consists of thousands of sub-pixel, semi-transparent strands. Treating hair as a fully opaque, solid surface creates harsh, binary shadows that eliminate the soft, light-scattering characteristics inherent to hair volumes.

To model hair accurately, light must attenuate gradually based on the Beer-Lambert law or accumulated strand opacity. The visibility function \(V(z)\) along a ray through a hair volume is fractional:

\[V(z) = \prod_{i=1}^{k} (1 - \alpha_i)\]

where \(\alpha_i\) represents the opacity of strand \(i\) encountered before reaching depth \(z\).

Transmittance Representation in GLSL

Deep shadow maps represent fractional transmittance as a monotonically decreasing piecewise linear function of depth: \(T(z) \in [0.0, 1.0]\). In modern GLSL, this continuous visibility curve is stored using one of several techniques depending on memory constraints:

  1. Per-Pixel Linked Lists (Uncompressed): Fragment shaders write pairs of depth and alpha values into a Shader Storage Buffer Object (SSBO) using atomic counters, sorting them by depth prior to lighting.
  2. Adaptive Volumetric Shadow Maps (AVSM): Compresses the curve into a fixed number of control points \((z_n, \tau_n)\) stored across multiple render targets (e.g., RGBA32F textures), dynamically merging control points that minimize error.
  3. Fourier Opacity Maps (FOM): Projects the transmittance curve into Fourier basis coefficients stored in an RGBA16F or RGBA32F 2D texture, avoiding sorting altogether.

Generation Phase: Building the Transmittance Function

During the light-space pass, hair geometry is rendered into light space. Rather than a standard depth-only pass, a GLSL fragment shader evaluates strand density and records visibility:

#version 460 core

layout(binding = 0, r32ui) uniform uimage2D headPointers;
layout(binding = 1, std430) buffer NodeBuffer {
    vec2 data[]; // x = depth, y = alpha
} nodes;

layout(binding = 0, offset = 0) uniform atomic_uint nodeCounter;

in float v_Depth;
in float v_Alpha;

void main() {
    uint newNodeIndex = atomicCounterIncrement(nodeCounter);
    uint prevIndex = imageAtomicExchange(headPointers, ivec2(gl_FragCoord.xy), newNodeIndex);

    // Pack depth and strand opacity for subsequent sorting and compression
    nodes.data[newNodeIndex] = vec2(v_Depth, v_Alpha);
}

Once all strands are submitted, a compute shader processes each light-space pixel column, sorts the nodes along the ray by depth, and pre-integrates total transmittance \(T(z)\).

Evaluation Phase: Sampling Transmittance in the Shading Pass

In the primary rendering pass from the camera's perspective, the hair fragment’s world position is transformed into light clip space. Instead of a boolean comparison (depth < lightDepth ? 1.0 : 0.0), GLSL searches the stored piecewise nodes to determine where the fragment sits along the transmittance curve.

When using an AVSM-style fixed control point array:

#version 460 core

uniform sampler2DArray u_DeepShadowDepths;
uniform sampler2DArray u_DeepShadowTransmittance;

in vec4 v_LightSpacePos;
out vec4 FragColor;

float EvaluateHairTransmittance(vec3 lightCoords) {
    // Current depth of fragment from the light
    float fragmentDepth = lightCoords.z;

    // Fetch control points (e.g., 4 nodes stored across texture layers)
    vec4 depths = texture(u_DeepShadowDepths, lightCoords.xy).rgba;
    vec4 trans  = texture(u_DeepShadowTransmittance, lightCoords.xy).rgba;

    // Fragment is in front of all hair occluders
    if (fragmentDepth <= depths.x) {
        return 1.0;
    }
    // Fragment is deeper than maximum recorded occluders
    if (fragmentDepth >= depths.w) {
        return trans.w;
    }

    // Piecewise linear interpolation between adjacent nodes
    for (int i = 0; i < 3; ++i) {
        if (fragmentDepth >= depths[i] && fragmentDepth < depths[i + 1]) {
            float t = (fragmentDepth - depths[i]) / (depths[i + 1] - depths[i]);
            return mix(trans[i], trans[i + 1], t);
        }
    }

    return 1.0;
}

void main() {
    vec3 projCoords = v_LightSpacePos.xyz / v_LightSpacePos.w;
    projCoords = projCoords * 0.5 + 0.5; // Map from [-1, 1] to [0, 1]

    float visibility = EvaluateHairTransmittance(projCoords);

    // Modulate direct lighting by fractional visibility
    vec3 baseColor = vec3(0.8, 0.6, 0.4);
    vec3 radiance = baseColor * visibility;

    FragColor = vec4(radiance, 1.0);
}

Filtering and Self-Shadowing Artifacts

Directly sampling a single deep shadow ray can introduce aliasing or acne along hair boundaries. In GLSL, this is mitigated by evaluating multiple jittered rays—similar to Percentage-Closer Filtering (PCF)—or by storing transmittance as continuous Fourier coefficients. Fourier Opacity Mapping replaces loop-based interpolation with direct trigonometric evaluation:

float EvaluateFourierTransmittance(vec4 a_k, vec4 b_k, float z) {
    float omega = 2.0 * 3.14159265 * z;
    vec4 cosVal = cos(vec4(1.0, 2.0, 3.0, 4.0) * omega);
    vec4 sinVal = sin(vec4(1.0, 2.0, 3.0, 4.0) * omega);
    
    // Compute the reconstructed optical depth
    float opticalDepth = dot(a_k, cosVal) + dot(b_k, sinVal);
    return clamp(exp(-opticalDepth), 0.0, 1.0);
}

Through piecewise reconstruction or frequency-domain approximation, GLSL evaluates the fractional survival of light through hair fibers, producing realistic volumetric shading and self-shadowing without rendering individual raymarched paths per camera pixel.