How Do Booleans and Ternary Operators Work in GLSL?

In OpenGL Shading Language (GLSL), boolean operations and conditional ternary operators provide essential logical control flow and value selection. While their syntax mirrors C-style languages, their execution model differs fundamentally due to the Single Instruction, Multiple Threads (SIMT) architecture of modern Graphics Processing Units (GPUs). This guide examines how GLSL evaluates boolean expressions and ternary operations, how the underlying hardware handles execution divergence through masking, and best practices for writing performant shader logic.

Boolean Types and Logical Operators in GLSL

GLSL includes a native scalar bool type as well as vector variations: bvec2, bvec3, and bvec4. Scalar booleans support standard logical operators:

  • Logical NOT (!): Inverts a boolean expression.
  • Logical AND (&&): Evaluates to true only if both operands are true.
  • Logical OR (||): Evaluates to true if at least one operand is true.
  • Logical XOR (^^): Evaluates to true if exactly one operand is true.

Relational operators such as <, <=, >, >=, ==, and != produce scalar booleans when comparing scalar types. When working with vector types, GLSL provides specialized built-in functions rather than component-wise logical operators:

  • lessThan(vecA, vecB), greaterThan(vecA, vecB): Return component-wise boolean vectors (bvec).
  • equal(vecA, vecB), notEqual(vecA, vecB): Produce bvec outputs comparing vector components.
  • any(bvec): Returns true if any component of the boolean vector is true.
  • all(bvec): Returns true only if all components are true.
  • not(bvec): Performs a component-wise inversion on a boolean vector.

The Conditional Ternary Operator

The conditional ternary operator (condition ? expression_true : expression_false) evaluates a scalar boolean condition and selects one of two expressions. Both return expressions must share the exact same type.

// Basic ternary syntax
float intensity = isLit ? 1.0 : 0.2;
vec3 finalColor = isHighlighted ? vec3(1.0, 0.0, 0.0) : baseColor;

Unlike scalar mathematics, the ternary operator in GLSL requires the condition to be a scalar bool. It cannot take a bvec directly; vector-wide selection is handled through functions such as mix().

// Component-wise conditional selection using mix
vec3 result = mix(colorA, colorB, vec3(conditionVec));

Execution Mechanics on the GPU

The physical execution of boolean logic and ternary operators depends on how GPU hardware processes parallel work units, typically grouped into warps (NVIDIA) or wavefronts (AMD).

Short-Circuit Evaluation

In standard C++, && and || guarantee short-circuit evaluation: if the left operand determines the outcome, the right operand is not evaluated. In GLSL, the specification allows short-circuiting for logical operators and the ternary operator, but hardware implementation varies:

  • Uniform Conditions: When all threads in a warp evaluate the same condition to the same branch, the GPU executes only the active branch path.
  • Divergent Conditions: When threads within the same warp evaluate conditions differently, the hardware must execute both execution paths sequentially while using internal execution masks to enable or disable writes for individual threads.

Predication and Execution Masking

For short instructions and ternary operators, modern GPU shader compilers often replace branch instructions with predication (conditional assignment).

Instead of jumping over instructions:

  1. The compiler evaluates the boolean condition and stores the result in an internal predicate register.
  2. The instructions for both the true and false expressions are computed.
  3. The hardware uses conditional select (csel or cmov) instructions to assign the final register values based on the predicate bit.

This predication mechanism avoids the high overhead of warp serialization and branch prediction penalties for simple scalar arithmetic.

Performance Considerations

  • Avoid Complex Branch Divergence: If expressions inside a ternary operator involve expensive operations (such as texture fetches or heavy trigonometry), divergence will force all threads in the warp to stall while both paths execute.
  • Utilize Arithmetic Step Functions: For pure numerical selections, built-in functions like step(), clamp(), and mix() can map directly to fast, vector-friendly instructions without creating execution divergence.
  • Derivative Safety: Functions that rely on implicit screen-space derivatives (such as texture() with mipmaps or dFdx/dFdy) must execute uniformly. Placing derivative-dependent lookups inside non-uniform ternary conditions can lead to undefined behavior or visual artifacts.