Screen-Space Refraction Raymarching in GLSL

Screen-space refraction (SSR) for non-planar meshes allows real-time rendering of complex, curved transparent objects by tracing refracted rays directly through the scene's existing depth and color buffers. Unlike simple 2D normal-offset distortions, raymarching screen-space depth evaluates the physical path of light through curved boundaries according to Snell’s Law, capturing realistic parallax, dynamic thickness, and internal occlusion. This article breaks down the mathematical formulation, view-space ray setup, linear and binary depth traversal, thickness validation, and GLSL implementation required to evaluate screen-space refraction for arbitrary non-planar geometry.

1. Refraction Ray Generation

Evaluating refraction on non-planar surfaces begins in the fragment shader by computing the incident vector and the surface normal in view space. Using view-space coordinates aligns the ray trajectory directly with the depth buffer's projection parameters.

Given the normalized fragment position in view space \(\mathbf{P}_{\text{view}}\) (where the eye is at the origin) and the normalized surface normal \(\mathbf{N}_{\text{view}}\), the incident vector is \(\mathbf{I} = \text{normalize}(\mathbf{P}_{\text{view}})\). The refraction vector \(\mathbf{R}\) is determined by Snell's Law using the ratio of indices of refraction \(\eta = n_{\text{air}} / n_{\text{medium}}\):

vec3 I = normalize(vViewPosition);
vec3 N = normalize(vViewNormal);
vec3 R = refract(I, N, etaRatio);

If total internal reflection occurs, refract() returns vec3(0.0). In this case, the shader must fall back to reflection raymarching or an environment map lookup.

2. Ray Parameterization and Coordinate Space

Raymarching can be executed directly in view space or projected into screen space (texture coordinate space). Tracing in view space with perspective-correct screen-space projection at each step yields uniform spatial steps, which prevents distortion artifacts on arbitrary 3D meshes:

vec3 rayOrigin = vViewPosition;
vec3 rayDir = normalize(R);
float stepSize = maxDistance / float(maxSteps);

For each step \(i\), the ray position is:

\[\mathbf{P}_i = \text{rayOrigin} + \mathbf{R} \cdot (i \cdot \text{stepSize})\]

To compare \(\mathbf{P}_i\) against the scene depth buffer, project \(\mathbf{P}_i\) onto the screen:

vec4 clipPos = projectionMatrix * vec4(P_i, 1.0);
vec3 ndc = clipPos.xyz / clipPos.w;
vec2 sampleUV = ndc.xy * 0.5 + 0.5;

3. Depth Buffer Sampling and Intersection Testing

The scene depth buffer—rendered in an opaque pre-pass containing all objects behind the transparent mesh—is sampled using sampleUV. Because raw depth values in non-linear clip space (\([0, 1]\) or \([-1, 1]\)) distort step intervals, both the ray depth and the sampled depth must be evaluated in linear view-space units (\(Z_{\text{view}}\)):

// Function to convert non-linear depth to linear view Z
float linearizeDepth(float depthSample, mat4 invProj) {
    float z_ndc = depthSample * 2.0 - 1.0;
    vec4 viewPos = invProj * vec4(0.0, 0.0, z_ndc, 1.0);
    return viewPos.z / viewPos.w;
}

An intersection occurs when the ray's linear depth becomes more negative (further from the camera in standard OpenGL coordinates) than the sampled scene depth:

\[\mathbf{P}_{i, z} \le Z_{\text{scene}}\]

Because the screen-space depth buffer only represents the front-most surface of opaque objects, a thickness threshold (\(\tau\)) must be enforced to avoid false collisions with geometry hidden behind the opaque surface:

float sceneDepth = linearizeDepth(texture(uDepthTexture, sampleUV).r, uInvProjMatrix);
float rayDepth = P_i.z;

// Check if ray is behind the surface, but within the geometry thickness
if (rayDepth <= sceneDepth && rayDepth >= sceneDepth - thicknessThreshold) {
    // Hit detected
}

Fixed-step linear marching causes stepping banding artifacts on curved geometries. Once a hit interval is detected between step \(t_{\text{prev}}\) and step \(t_{\text{curr}}\), a binary search refines the exact surface intersection point:

vec3 minPoint = P_prev;
vec3 maxPoint = P_curr;

for (int j = 0; j < 8; ++j) {
    vec3 midPoint = mix(minPoint, maxPoint, 0.5);
    vec4 clip = projectionMatrix * vec4(midPoint, 1.0);
    vec2 uv = (clip.xy / clip.w) * 0.5 + 0.5;
    
    float sceneZ = linearizeDepth(texture(uDepthTexture, uv).r, uInvProjMatrix);
    
    if (midPoint.z <= sceneZ) {
        maxPoint = midPoint; // Ray is behind geometry; pull back
    } else {
        minPoint = midPoint; // Ray is in front; push forward
    }
}

The resulting minPoint or maxPoint provides the sub-pixel UV coordinates for sampling the refracted scene color.

5. GLSL Raymarching Implementation

The following complete fragment routine marches through the scene depth buffer and resolves the final refracted color:

uniform sampler2D uSceneColor;
uniform sampler2D uDepthTexture;
uniform mat4 projectionMatrix;
uniform mat4 uInvProjMatrix;

vec3 evaluateRefraction(vec3 viewPos, vec3 viewNormal, float eta) {
    vec3 I = normalize(viewPos);
    vec3 N = normalize(viewNormal);
    vec3 R = refract(I, N, eta);

    if (length(R) == 0.0) {
        // Total internal reflection fallback
        return texture(uSceneColor, gl_FragCoord.xy / vec2(textureSize(uSceneColor, 0))).rgb;
    }

    const int MAX_STEPS = 64;
    const int BINARY_STEPS = 8;
    const float MAX_DISTANCE = 10.0;
    const float THICKNESS = 0.5;

    float stepSize = MAX_DISTANCE / float(MAX_STEPS);
    vec3 currentPos = viewPos;
    vec3 lastPos = viewPos;
    vec2 hitUV = vec2(0.0);
    bool hitFound = false;

    for (int i = 0; i < MAX_STEPS; ++i) {
        currentPos += R * stepSize;

        vec4 clip = projectionMatrix * vec4(currentPos, 1.0);
        vec2 uv = (clip.xy / clip.w) * 0.5 + 0.5;

        // Abort if the ray exits screen boundaries
        if (uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0) {
            break;
        }

        float sceneZ = linearizeDepth(texture(uDepthTexture, uv).r, uInvProjMatrix);

        if (currentPos.z <= sceneZ && currentPos.z >= sceneZ - THICKNESS) {
            // Binary search refinement
            vec3 minP = lastPos;
            vec3 maxP = currentPos;

            for (int j = 0; j < BINARY_STEPS; ++j) {
                vec3 mid = (minP + maxP) * 0.5;
                vec4 midClip = projectionMatrix * vec4(mid, 1.0);
                vec2 midUV = (midClip.xy / midClip.w) * 0.5 + 0.5;
                float midSceneZ = linearizeDepth(texture(uDepthTexture, midUV).r, uInvProjMatrix);

                if (mid.z <= midSceneZ) {
                    maxP = mid;
                } else {
                    minP = mid;
                }
            }

            vec4 finalClip = projectionMatrix * vec4(maxP, 1.0);
            hitUV = (finalClip.xy / finalClip.w) * 0.5 + 0.5;
            hitFound = true;
            break;
        }
        lastPos = currentPos;
    }

    if (hitFound) {
        return texture(uSceneColor, hitUV).rgb;
    }

    // Fallback: regular screen-space UV
    vec2 fallbackUV = gl_FragCoord.xy / vec2(textureSize(uSceneColor, 0));
    return texture(uSceneColor, fallbackUV).rgb;
}

6. Edge Falloff and Occlusion Artifact Handling

Because screen-space raymarching is constrained by viewport bounds, rays exiting the screen or hitting untracked occluded regions produce visual discontinuities. To smooth these transitions:

Screen-Edge Vignette: Fade the refracted color to the unrefracted background sample as hitUV approaches the screen borders:

vec2 edgeFactor = smoothstep(0.0, 0.05, hitUV) * (1.0 - smoothstep(0.95, 1.0, hitUV));
float vignette = edgeFactor.x * edgeFactor.y;
return mix(fallbackColor, hitColor, vignette);

Dynamic Step Scaling: Scale stepSize based on the incident angle relative to the surface normal (\(\mathbf{N} \cdot \mathbf{I}\)). Grazing angles require smaller step sizes to maintain accuracy across steep, non-planar gradients.