GLSL layout(cw) vs layout(ccw) Vertex Winding Order

In GLSL tessellation shaders, the layout(cw) and layout(ccw) qualifiers define the vertex winding order—clockwise or counter-clockwise—of primitives generated by the tessellation stage. Declared within the Tessellation Evaluation Shader (TES), these layout qualifiers dictate the sequence in which generated vertices are assembled into triangles, directly impacting face culling and front-face determination in the OpenGL rendering pipeline.

The Role of the Primitive Generator

During the tessellation process, the Tessellation Control Shader (TCS) outputs tessellation levels, which the fixed-function Primitive Generator (PG) uses to subdivide an abstract patch into smaller geometric primitives (such as triangles or quads).

Because the generator operates on normalized coordinates \((u, v, w)\), it must know how to index the vertices of each newly created polygon. The winding order qualifiers in the TES tell the primitive generator how to order these vertices.

layout(ccw) (Counter-Clockwise)

  • Behavior: Instructs the primitive generator to emit vertices in a counter-clockwise sequence when viewed from the front of the patch.
  • Default Status: ccw is the default winding order in GLSL tessellation evaluation shaders if no ordering qualifier is specified.
  • Pipeline Alignment: In OpenGL, the default front-facing polygon orientation is counter-clockwise (glFrontFace(GL_CCW)). Using layout(ccw) keeps your tessellated surfaces consistent with standard OpenGL conventions.
layout(triangles, equal_spacing, ccw) in;

layout(cw) (Clockwise)

  • Behavior: Instructs the primitive generator to emit vertices in a clockwise sequence when viewed from the front of the patch.
  • Pipeline Alignment: If your application sets glFrontFace(GL_CW), or if the patch topology requires reversed vertex ordering to be considered front-facing, specifying layout(cw) ensures the triangles match this expectation.
layout(triangles, equal_spacing, cw) in;

Impact on the Graphics Pipeline

  1. Face Culling: The primary function affected by this qualifier is OpenGL face culling (glEnable(GL_CULL_FACE)). If a shader declares layout(cw) while the pipeline expects GL_CCW for front faces, the rasterizer will classify visible front faces as back faces, causing them to be discarded when GL_BACK culling is active.
  2. Normal Calculation: If surface normals are computed dynamically in the evaluation or fragment stage using cross products of adjacent edge vectors (e.g., cross(v1 - v0, v2 - v0)), an unexpected winding order flips the direction of the normal vector, which can invert lighting and shading calculations.