How Non-Square Matrices Work in GLSL

This article provides an overview of how non-square matrix types like mat2x3 and mat4x3 function within the OpenGL Shading Language (GLSL). It explains the column-major naming convention, internal memory layout, component indexing, rules for matrix-vector and matrix-matrix multiplication, and standard graphics use cases such as memory-efficient skinning transforms.

Dimension Syntax: Columns by Rows

In GLSL, matrix types follow the notation matCxR, where C represents the number of columns and R represents the number of rows. This can be counterintuitive because standard mathematical notation lists dimensions as \(\text{rows} \times \text{columns}\).

  • mat2x3 has 2 columns and 3 rows.
  • mat4x3 has 4 columns and 3 rows.
  • mat3x2 has 3 columns and 2 rows.

If only a single digit is supplied (e.g., mat3), GLSL treats it as a square matrix (mat3x3).

Memory Layout and Indexing

GLSL matrices are column-major. A non-square matrix is treated internally as an array of column vectors.

For example, a mat4x3 is a collection of four vec3 column vectors:

mat4x3 m;
vec3 firstColumn  = m[0]; // Accesses the first column
vec3 fourthColumn = m[3]; // Accesses the fourth column

To access an individual scalar element, use the syntax m[column][row]:

float val = m[2][1]; // Column index 2, Row index 1

Constructors accept components in column-major order as well. You can initialize a mat2x3 using two vec3 variables or six individual floats:

// Using column vectors
mat2x3 m1 = mat2x3(vec3(1.0, 2.0, 3.0), vec3(4.0, 5.0, 6.0));

// Using scalar values (filled column-by-column)
mat2x3 m2 = mat2x3(
    1.0, 2.0, 3.0,  // Column 0
    4.0, 5.0, 6.0   // Column 1
);

Vector and Matrix Multiplication Rules

Matrix multiplication in GLSL requires the inner dimensions of the operands to match, exactly as in standard linear algebra.

Matrix-Vector Multiplication (Matrix * Vector)

When multiplying a matrix by a vector on the right, the vector behaves as a column vector. The number of components in the vector must equal the number of columns in the matrix. The result is a vector with the size equal to the matrix's row count:

\[\text{matCxR} \times \text{vecC} = \text{vecR}\]

  • mat4x3 * vec4 yields a vec3.
  • mat2x3 * vec2 yields a vec3.
mat4x3 transform;
vec4 localPosition = vec4(x, y, z, 1.0);
vec3 worldPosition = transform * localPosition; // Valid: (4x3) * (4x1) = (3x1)

Vector-Matrix Multiplication (Vector * Matrix)

When the vector is on the left, it acts as a row vector. The vector's size must match the number of rows in the matrix:

\[\text{vecR} \times \text{matCxR} = \text{vecC}\]

  • vec3 * mat4x3 yields a vec4.

Matrix-Matrix Multiplication

For two matrices \(A \times B\), the number of columns in \(A\) must equal the number of rows in \(B\):

\[\text{matAxB} \times \text{matCxA} = \text{matCxB}\]

  • Multiplying a mat2x3 (2 columns, 3 rows) by a mat4x2 (4 columns, 2 rows) produces a mat4x3 (4 columns, 3 rows).

Common Use Cases

The most frequent application of non-square matrices in shaders is the mat4x3. Standard 3D affine transformations (rotation, scaling, translation) are typically stored in \(4 \times 4\) matrices where the bottom row is always [0, 0, 0, 1].

By dropping this redundant fourth row and using a mat4x3, shaders achieve two advantages:

  1. Memory Reduction: Storing transformation palettes for skeletal animation (skinning) in uniform buffers requires 25% less data bandwidth (12 floats per bone instead of 16).
  2. Computational Efficiency: Multiplying a mat4x3 by a homogeneous vec4 position directly computes the resulting 3D cartesian coordinates (vec3) without performing the trivial arithmetic operations of the fourth row.