How gl_TessLevelInner Controls GLSL Tessellation Density

This article explains how the built-in gl_TessLevelInner array governs interior mesh subdivision in OpenGL Shading Language (GLSL). It details the array's structure within the Tessellation Control Shader (TCS), how its floating-point values determine the density of internal vertices across quad and triangle patch primitives, and how it interacts with different primitive spacing modes in the tessellation pipeline.

The Role of gl_TessLevelInner

In the GLSL Tessellation Control Shader (TCS), gl_TessLevelInner is a built-in output array defined as:

patch out float gl_TessLevelInner[2];

While gl_TessLevelOuter determines how many segments are generated along the perimeter edges of an input patch, gl_TessLevelInner dictates how the interior region of that patch is subdivided. The fixed-function primitive generator uses these inner levels to populate vertices inside the patch, bridging the internal mesh structure to the outer perimeter edges.

Behavior Across Patch Types

The exact interpretation of gl_TessLevelInner depends directly on the abstract patch domain declared in the Tessellation Evaluation Shader (TES)—specifically quads, triangles, or isolines.

1. Quad Domain (quads)

For quad patches, the interior is subdivided across a two-dimensional grid, utilizing both elements of the array:

  • gl_TessLevelInner[0]: Determines the tessellation density along the horizontal axis (the \(u\)-direction). It sets how many subdivisions exist between the left and right boundary edges.
  • gl_TessLevelInner[1]: Determines the tessellation density along the vertical axis (the \(v\)-direction). It sets how many subdivisions exist between the top and bottom boundary edges.

The tessellation generator creates an internal rectangular grid based on these two values and automatically generates triangle strips connecting the inner grid points to the outer boundary vertices defined by gl_TessLevelOuter.

2. Triangle Domain (triangles)

For triangular patches, the interior is subdivided using concentric triangular rings:

  • gl_TessLevelInner[0]: Specifies the subdivision density for the entire interior. This value controls the number of nested concentric triangles generated between the outer boundary and the patch's centroid.
  • gl_TessLevelInner[1]: Completely ignored for triangular patches.

When gl_TessLevelInner[0] is equal to 1, the patch has no concentric inner triangles, and the outer edges connect directly to a single interior point or form a minimal triangulation. As the value increases, more nested triangular rings are formed, creating a higher density of internal polygons.

3. Isoline Domain (isolines)

For isoline patches, gl_TessLevelInner is entirely unused. The primitive generator only evaluates gl_TessLevelOuter[0] (the number of lines) and gl_TessLevelOuter[1] (the number of segments per line). Setting values in gl_TessLevelInner has no effect on isolines.

Culling and Primitive Spacing

The floating-point values assigned to gl_TessLevelInner interact directly with edge culling and spacing modes:

  • Patch Discarding: If any value in gl_TessLevelInner (or gl_TessLevelOuter) evaluates to less than or equal to zero (or NaN), the primitive generator discards the patch completely before it reaches the Tessellation Evaluation Shader.
  • Spacing Qualifiers: The primitive generator processes inner levels according to the spacing mode declared in the TES:
    • equal_spacing: The inner level is clamped to the range \([1, \text{GL\_MAX\_TESS\_GEN\_LEVEL}]\) and rounded up to the nearest integer.
    • fractional_even_spacing: Clamped to \([2, \text{GL\_MAX\_TESS\_GEN\_LEVEL}]\) and rounded up to the next even integer, with segments scaling smoothly as the fractional value changes.
    • fractional_odd_spacing: Clamped to \([1, \text{GL\_MAX\_TESS\_GEN\_LEVEL} - 1]\) and rounded up to the next odd integer, allowing smooth dynamic transitions between levels of detail without mesh popping.