Vector Projection Formula in GLSL

Vector projection is a fundamental mathematical operation used in 3D graphics, shading, and physics simulation to determine the orthogonal shadow of one vector onto another. In the OpenGL Shading Language (GLSL), there is no built-in project() function, but the operation can be implemented cleanly and efficiently using the native dot() function. This article outlines the mathematical formula for vector projection and shows how to write it directly in GLSL shaders for arbitrary or normalized vectors.

The Mathematical Projection Formula

To project a vector \(\mathbf{a}\) onto a target vector \(\mathbf{b}\), the projection vector \(\text{proj}_{\mathbf{b}}(\mathbf{a})\) is defined as:

\[\text{proj}_{\mathbf{b}}(\mathbf{a}) = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{b}\|^2} \mathbf{b}\]

Because the squared length of a vector \(\|\mathbf{b}\|^2\) is equivalent to the dot product of the vector with itself (\(\mathbf{b} \cdot \mathbf{b}\)), the formula simplifies to:

\[\text{proj}_{\mathbf{b}}(\mathbf{a}) = \left(\frac{\mathbf{a} \cdot \mathbf{b}}{\mathbf{b} \cdot \mathbf{b}}\right) \mathbf{b}\]

If vector \(\mathbf{b}\) is already a unit vector (normalized to a length of 1), the denominator equals 1, simplifying the operation to:

\[\text{proj}_{\hat{\mathbf{b}}}(\mathbf{a}) = (\mathbf{a} \cdot \hat{\mathbf{b}}) \hat{\mathbf{b}}\]

GLSL Implementation

General Case (Arbitrary Vectors)

When the vector being projected onto is not guaranteed to be normalized, compute the projection using the dot product for both the numerator and the denominator. Using dot(b, b) avoids expensive square root operations associated with finding vector lengths.

vec3 project(vec3 a, vec3 b) {
    return (dot(a, b) / dot(b, b)) * b;
}

This function can be adapted to vec2 or vec4 by simply replacing the type definitions, as GLSL overloads dot() for all floating-point vector types.

Optimized Case (Normalized Target Vector)

If the target vector is already normalized (such as a surface normal or directional vector), the calculation reduces to a single dot product and a scalar-vector multiplication:

vec3 projectOnUnit(vec3 a, vec3 b_norm) {
    return dot(a, b_norm) * b_norm;
}

Vector Rejection (Orthogonal Component)

In many shader calculations, such as Gram-Schmidt orthogonalization or computing tangential forces, it is necessary to obtain the component of \(\mathbf{a}\) perpendicular to \(\mathbf{b}\). This perpendicular vector (the rejection) is obtained by subtracting the projected vector from the original vector:

vec3 reject(vec3 a, vec3 b) {
    return a - project(a, b);
}