Quaternion to 3x3 Rotation Matrix in GLSL

Converting a unit quaternion to a 3x3 rotation matrix in GLSL enables shaders to apply rotations to 3D geometry smoothly while avoiding gimbal lock. This article details the mathematical formulation behind the transformation, addresses GLSL's column-major matrix ordering, and provides an optimized GLSL function to construct a mat3 from a vec4 representation of a quaternion.

Mathematical Formulation

A unit quaternion \(q\) is defined by its components \((x, y, z, w)\), where \(\mathbf{v} = (x, y, z)\) represents the vector (imaginary) part and \(w\) represents the scalar (real) part, satisfying the condition \(x^2 + y^2 + z^2 + w^2 = 1\).

The corresponding \(3 \times 3\) rotation matrix \(R\) is defined algebraically as:

\[ R = \begin{bmatrix} 1 - 2(y^2 + z^2) & 2(xy - wz) & 2(xz + wy) \\ 2(xy + wz) & 1 - 2(x^2 + z^2) & 2(yz - wx) \\ 2(xz - wy) & 2(yz + wx) & 1 - 2(x^2 + y^2) \end{bmatrix} \]

GLSL Matrix Storage Convention

GLSL constructs matrices using column-major order. When passing individual float components to the mat3 constructor, arguments populate column 0 first, followed by column 1, then column 2. Alternatively, you can supply three vec3 arguments, each defining a complete column:

• Column 0: (1 - 2(y² + z²), 2(xy + wz), 2(xz - wy))
• Column 1: (2(xy - wz), 1 - 2(x² + z²), 2(yz + wx))
• Column 2: (2(xz + wy), 2(yz - wx), 1 - 2(x² + y²))

GLSL Implementation

Below is a GLSL function that converts a normalized quaternion into a mat3. Common subexpressions are precalculated to minimize arithmetic operations on the GPU.

mat3 quatToMat3(vec4 q) {
    // Assumes q is a normalized quaternion: vec4(x, y, z, w)
    float qx2 = q.x + q.x;
    float qy2 = q.y + q.y;
    float qz2 = q.z + q.z;

    float xx = q.x * qx2;
    float yy = q.y * qy2;
    float zz = q.z * qz2;

    float xy = q.x * qy2;
    float xz = q.x * qz2;
    float yz = q.y * qz2;

    float wx = q.w * qx2;
    float wy = q.w * qy2;
    float wz = q.w * qz2;

    return mat3(
        // Column 0
        1.0 - (yy + zz),
        xy + wz,
        xz - wy,

        // Column 1
        xy - wz,
        1.0 - (xx + zz),
        yz + wx,

        // Column 2
        xz + wy,
        yz - wx,
        1.0 - (xx + yy)
    );
}

Considerations

• Normalization: The formula assumes the quaternion has a length of 1. If the quaternion undergoes linear interpolation (such as mix) prior to conversion, run normalize(q) before passing it to the function to prevent distortion and non-orthogonal scale artifacts.
• Vector Transformation: With the matrix returned in column-major format, rotating a vector \(\mathbf{v}\) is performed using standard matrix multiplication: vec3 rotatedVec = quatToMat3(q) * v;.