GLSL layout(quads) UV Domain Generation Explained

In a GLSL tessellation evaluation shader (TES), specifying the layout(quads) input layout qualifier instructs the primitive generator to create a two-dimensional grid over a normalized square domain. This overview explains how layout(quads) redefines the abstract UV coordinate space, dictates the format and values of the built-in gl_TessCoord variable, changes the mapping of inner and outer tessellation levels, and enables standard bilinear interpolation instead of barycentric coordinate math.

The 2D Rectangular Domain

When an input primitive is declared with layout(quads), the fixed-function primitive generation stage constructs a tessellated domain defined over the unit square \([0.0, 1.0] \times [0.0, 1.0]\). This is fundamentally different from layout(triangles), which generates an equilateral triangle domain governed by barycentric coordinates. The quad domain represents a continuous UV space with four distinct boundaries:

• \(u = 0.0\) (Left edge)
• \(u = 1.0\) (Right edge)
• \(v = 0.0\) (Bottom edge)
• \(v = 1.0\) (Top edge)

Modification of gl_TessCoord

The evaluation shader receives the location of each generated vertex within this domain through the built-in vector gl_TessCoord. Under layout(quads), the components of gl_TessCoord behave as follows:

• gl_TessCoord.x: Represents the normalized \(u\)-coordinate, ranging from \(0.0\) to \(1.0\).
• gl_TessCoord.y: Represents the normalized \(v\)-coordinate, ranging from \(0.0\) to \(1.0\).
• gl_TessCoord.z: Is unused and always set to \(0.0\).

Because only the X and Y components vary, shaders evaluate quad topology using standard two-dimensional Cartesian coordinates (gl_TessCoord.xy).

Mapping Tessellation Levels to UV Axes

The distribution and density of generated UV coordinates are governed by the tessellation levels calculated in the tessellation control shader (TCS):

• Inner Levels (gl_TessLevelInner): A quad uses two inner tessellation levels. gl_TessLevelInner[0] specifies the number of subdivisions along the vertical (\(v\)) axis, while gl_TessLevelInner[1] specifies the subdivisions along the horizontal (\(u\)) axis. Together, they define the interior grid density of UV samples.
• Outer Levels (gl_TessLevelOuter): A quad uses all four outer levels. Each index directly corresponds to one of the four perimeter edges of the \([0, 1]\) domain:
  • gl_TessLevelOuter[0]: Subdivides the edge where \(u = 0.0\).
  • gl_TessLevelOuter[1]: Subdivides the edge where \(v = 0.0\).
  • gl_TessLevelOuter[2]: Subdivides the edge where \(u = 1.0\).
  • gl_TessLevelOuter[3]: Subdivides the edge where \(v = 1.0\).

Transitioning between the interior grid and perimeter boundaries creates non-uniform UV coordinate stepping near the edges to match neighboring patches with different tessellation factors.

Vertex Position Interpolation

Because layout(quads) produces independent \(u\) and \(v\) coordinates rather than a 3-component barycentric weight vector, vertex attributes such as position, normals, and texture coordinates are computed via bilinear interpolation or tensor product surfaces (such as bicubic Bézier patches). In a standard four-control-point quad patch, bilinear interpolation is applied directly using gl_TessCoord.x and gl_TessCoord.y:

vec4 p0 = mix(gl_in[0].gl_Position, gl_in[1].gl_Position, gl_TessCoord.x);
vec4 p1 = mix(gl_in[3].gl_Position, gl_in[2].gl_Position, gl_TessCoord.x);
gl_Position = mix(p0, p1, gl_TessCoord.y);

By switching to layout(quads), the tessellation evaluation stage transitions from simplex-based coordinate weighting to structured 2D UV mapping, making it the preferred primitive configuration for terrain grids, displacement mapping, and parametric surfaces.