GTAO Sector Integration Formulation in GLSL

Ground Truth Ambient Occlusion (GTAO) improves upon standard screen-space ambient occlusion by formulating visibility as an exact spherical integral over directional slices. The technique decomposes the three-dimensional hemispherical ambient occlusion integral into a series of two-dimensional slice planes aligned with the view vector. Within each slice, ray-marched depth samples determine two maximum horizon angles, allowing the cosine-weighted occlusion integral between those horizons to be evaluated using a closed-form analytical solution. This article outlines the mathematical derivation of this sector integration and provides a direct, production-ready GLSL implementation.

Mathematical Formulation

The standard ambient occlusion equation at a surface point with unit normal \(\mathbf{n}\) evaluates the visibility function \(V(\omega)\) over the upper hemisphere \(\Omega\):

\[A = \frac{1}{\pi} \int_{\Omega} V(\omega) \max(0, \mathbf{n} \cdot \omega) \, d\omega\]

GTAO decomposes this integral into \(M\) rotating 2D planar slices around the view vector \(\mathbf{v} = (0, 0, 1)\) across angular directions \(\phi \in [0, \pi)\). For a given slice at angle \(\phi\), the directions on the slice circle are parameterized by angle \(\theta \in [-\frac{\pi}{2}, \frac{\pi}{2}]\) relative to \(\mathbf{v}\).

The surface normal \(\mathbf{n}\) is projected onto the slice plane:

Direction vector of the slice in screen space: \(\mathbf{u}_\phi = (\cos\phi, \sin\phi, 0)\)
Projected components: \(n_t = \mathbf{n} \cdot \mathbf{u}_\phi\) and \(n_v = \mathbf{n} \cdot \mathbf{v}\)
Projected normal magnitude: \(|\mathbf{n}_p| = \sqrt{n_t^2 + n_v^2}\)
Angle of the projected normal relative to \(\mathbf{v}\): \(\gamma = \operatorname{atan2}(n_t, n_v)\)

Within the slice, the dot product simplifies via the cosine difference identity:

\[\mathbf{n} \cdot \omega(\theta) = |\mathbf{n}_p| \cos(\theta - \gamma)\]

Because spherical coordinates around the view direction yield the solid angle element \(d\omega = \sin\theta \, d\theta \, d\phi\), the inner angular integral along a single slice between two horizon angles \(h_1\) and \(h_2\) is:

\[I(h_1, h_2) = |\mathbf{n}_p| \int_{h_1}^{h_2} \cos(\theta - \gamma) \sin\theta \, d\theta\]

Using the trigonometric product-to-sum identity \(\cos(\theta - \gamma)\sin\theta = \frac{1}{2}[\sin(2\theta - \gamma) + \sin\gamma]\), the antiderivative \(F(\theta)\) evaluates directly to:

\[F(\theta) = \int \left[ \frac{1}{2} \sin(2\theta - \gamma) + \frac{1}{2} \sin\gamma \right] d\theta = \frac{1}{4} \left( 2\theta \sin\gamma - \cos(2\theta - \gamma) \right)\]

Evaluating across the horizon interval \([h_1, h_2]\) yields the exact closed-form contribution for the slice:

\[I(h_1, h_2) = |\mathbf{n}_p| \cdot \left[ F(h_2) - F(h_1) \right]\]

Horizon Angle Clamping

The searched horizons \(h_1\) (negative direction, \(h_1 \in [-\frac{\pi}{2}, 0]\)) and \(h_2\) (positive direction, \(h_2 \in [0, \frac{\pi}{2}]\)) must be clamped to the tangent plane of the surface to avoid integrating below the surface horizon. The local hemisphere boundary relative to the projected normal angle \(\gamma\) is \([\gamma - \frac{\pi}{2}, \gamma + \frac{\pi}{2}]\).

The clamped horizon angles are:

\[h_1 = \max\left(h_1, \, \gamma - \frac{\pi}{2}\right)\] \[h_2 = \min\left(h_2, \, \gamma + \frac{\pi}{2}\right)\]

GLSL Implementation

The following GLSL functions implement the projected normal calculation, horizon clamping, and closed-form sector integration.

const float PI = 3.14159265359;
const float HALF_PI = 1.57079632679;

// Evaluates the antiderivative F(theta) = 0.25 * (2.0 * theta * sin(gamma) - cos(2.0 * theta - gamma))
float GTAO_Antiderivative(float theta, float gamma) {
    return 0.25 * (2.0 * theta * sin(gamma) - cos(2.0 * theta - gamma));
}

// Computes the sector integration for a single slice
// h1: negative horizon angle in radians [-PI/2, 0]
// h2: positive horizon angle in radians [0, PI/2]
// viewSpaceNormal: normalized normal vector in view space (v = vec3(0, 0, 1))
// sliceDir: normalized 2D direction (cos(phi), sin(phi))
float GTAO_IntegrateSlice(float h1, float h2, vec3 viewSpaceNormal, vec2 sliceDir) {
    // Project view-space normal onto the 2D slice plane
    float n_t = dot(viewSpaceNormal.xy, sliceDir);
    float n_v = viewSpaceNormal.z;
    float projNormLen = length(vec2(n_t, n_v));

    if (projNormLen < 1e-6) {
        return 0.0;
    }

    // Angle gamma between the projected normal and the view axis (0, 0, 1)
    float gamma = atan(n_t, n_v);

    // Clamp horizons to the surface hemisphere defined by gamma
    h1 = max(h1, gamma - HALF_PI);
    h2 = min(h2, gamma + HALF_PI);

    // Integrate visibility between horizon bounds
    float visibility = GTAO_Antiderivative(h2, gamma) - GTAO_Antiderivative(h1, gamma);
    
    // Scale by projected normal magnitude
    return projNormLen * max(0.0, visibility);
}

To compute the final ambient occlusion coefficient, average GTAO_IntegrateSlice across all sampled slices \(M\), multiply by \(2\) (accounting for the \(\frac{1}{\pi} \int_0^\pi d\phi\) normalization factor where the slice spans \([-\pi/2, \pi/2]\)), and subtract the resulting visibility from \(1.0\) to produce the final occlusion factor.