GLSL Geometry Shader Input Primitive Layouts

In the OpenGL rendering pipeline, the geometry shader sits between the vertex shader and the rasterizer, processing complete geometric primitives rather than individual vertices. This article provides a comprehensive guide to the input primitive layout types supported by the OpenGL Shading Language (GLSL), detailing the required declarations, expected vertex counts, mapping to OpenGL draw modes, and practical applications for each type.

To receive geometry data, a GLSL geometry shader must declare an input layout qualifier at global scope using the following syntax:

layout (primitive_type) in;

GLSL supports five distinct input primitive layout types:

1. points

  • Vertex Count: 1 (gl_in[0])
  • Compatible OpenGL Primitives: GL_POINTS
  • Description: The geometry shader operates on single, isolated vertices.
  • Common Use Cases: Particle systems, generating camera-facing billboards/quads from point clouds, point sprite expansion, and dynamic point-based effects.

2. lines

  • Vertex Count: 2 (gl_in[0] to gl_in[1])
  • Compatible OpenGL Primitives: GL_LINES, GL_LINE_STRIP, GL_LINE_LOOP
  • Description: The shader processes segments consisting of two vertices forming a start point and an end point. When using line strips or loops, the hardware automatically decomposes the strip into individual two-vertex pairs for the geometry shader.
  • Common Use Cases: Extruding lines into 3D cylinders, generating volumetric hair or fur strands, screen-space thick line rendering, and vector field visualization.

3. lines_adjacency

  • Vertex Count: 4 (gl_in[0] to gl_in[3])
  • Compatible OpenGL Primitives: GL_LINES_ADJACENCY, GL_LINE_STRIP_ADJACENCY
  • Description: Provides four vertices per primitive. The central line segment is defined by vertices gl_in[1] and gl_in[2], while gl_in[0] and gl_in[3] represent the adjacent neighboring vertices preceding and following the segment.
  • Common Use Cases: Smooth spline interpolation, computing mitered joints and caps for wide polyline rendering, and continuous line subdivision.

4. triangles

  • Vertex Count: 3 (gl_in[0] to gl_in[2])
  • Compatible OpenGL Primitives: GL_TRIANGLES, GL_TRIANGLE_STRIP, GL_TRIANGLE_FAN
  • Description: The shader accepts three vertices representing a single triangle face. Strip and fan topologies are unpacked into discrete three-vertex primitives prior to invocation.
  • Common Use Cases: Calculating flat face normals on the fly, mesh wireframe overlays via barycentric coordinates, triangle-level frustum culling, and explosive mesh fragmentation effects.

5. triangles_adjacency

  • Vertex Count: 6 (gl_in[0] to gl_in[5])
  • Compatible OpenGL Primitives: GL_TRIANGLES_ADJACENCY, GL_TRIANGLE_STRIP_ADJACENCY
  • Description: Delivers the primary triangle along with its three neighboring vertices that share edges. Vertices gl_in[0], gl_in[2], and gl_in[4] form the main triangle, while gl_in[1], gl_in[3], and gl_in[5] provide the adjacent vertices opposite to edges (0-2), (2-4), and (4-0) respectively.
  • Common Use Cases: Silhouette edge detection for stencil shadow volumes, curvature estimation, cartoon edge outlining, and crack-free adaptive displacement.

Execution Considerations

The primitive layout declared in the geometry shader must strictly match the primitive topology submitted by the host application via draw calls (such as glDrawArrays or glDrawElements). Submitting an incompatible primitive type—for example, drawing GL_TRIANGLES when the geometry shader specifies layout (lines) in—will trigger an OpenGL GL_INVALID_OPERATION error.