GLSL 16-Bit Half-Float: Triggers for Inf and NaN

In GLSL, utilizing 16-bit floating-point numbers (mediump or explicit float16_t via extensions) improves shader execution speed and reduces register pressure, but it dramatically narrows the dynamic range compared to standard 32-bit floats. This article examines the exact arithmetic scenarios, function domain violations, and hardware behaviors that trigger Infinity (Inf) and Not-a-Number (NaN) outputs in 16-bit floating-point GLSL operations.

Understanding the 16-Bit Half-Float Constraints

Standard 16-bit half-precision floats follow the IEEE 754 binary16 standard, allocating 1 bit for the sign, 5 bits for the exponent, and 10 bits for the mantissa (fraction). This structure imposes strict mathematical limits:

  • Maximum finite value: \(65,504\)
  • Minimum normalized positive value: \(2^{-14} \approx 6.10 \times 10^{-5}\)
  • Smallest subnormal value: \(2^{-24} \approx 5.96 \times 10^{-8}\)

Because of this narrow threshold, operations that are completely safe in 32-bit float (highp) easily breach binary16 limits and produce exceptional values.

Conditions That Trigger Inf (Infinity)

In GLSL, +Inf or -Inf occurs when a computation generates a magnitude that exceeds the representable range or involves an asymptotic limit.

1. Arithmetic Overflow

Any mathematical operation yielding an absolute value greater than \(65,504\) causes an overflow. By IEEE 754 default rounding modes, this rounds to signed infinity:

  • Multiplication and Squaring: Multiplying two numbers whose product exceeds \(65,504\) (for example, \(256.0 \times 256.0 = 65,536\), immediately returning +Inf).
  • Summation Accumulators: Accumulating lighting loops, high-iteration raymarch steps, or high-dynamic-range (HDR) color values without clamping.

2. Division by Zero

Dividing any non-zero, finite 16-bit float by an exact zero (\(+0.0\) or \(-0.0\)) yields signed infinity:

  • x / 0.0 returns +Inf (if \(x > 0\)) or -Inf (if \(x < 0\)).

3. Mathematical Functions Exceeding Range

Several GLSL built-in functions quickly exceed \(65,504\) under small inputs:

  • exp(x) and exp2(x): Calling exp(x) where \(x > \ln(65504) \approx 11.09\) produces +Inf. For exp2(x), any \(x \ge 16.0\) overflows immediately.
  • pow(base, exp): Combinations where \(base^{exp} > 65,504\) (e.g., specular highlights such as pow(NdotH, 64.0) when the base is not strictly bounded below \(1.0\)).
  • inversesqrt(x) / 1.0 / sqrt(x): Passing an extremely small subnormal value or exact positive zero produces +Inf.

Conditions That Trigger NaN (Not a Number)

A NaN is produced when an operation has an undefined mathematical result or when invalid inputs are passed to built-in GLSL intrinsics.

1. Indeterminate Arithmetic Forms

Standard indeterminate operations immediately evaluate to NaN:

  • 0.0 / 0.0 (Zero divided by zero).
  • Inf / Inf or -Inf / Inf (Division of infinities).
  • 0.0 * Inf or 0.0 * -Inf (Zero multiplied by infinity).
  • Inf - Inf or Inf + (-Inf) (Addition of opposing infinities).

2. Domain Errors in Built-In Functions

Providing parameters outside a function's defined mathematical domain triggers a NaN:

  • sqrt(x) and inversesqrt(x): Supplying any value where \(x < 0.0\).
  • log(x) and log2(x): Supplying any value where \(x < 0.0\). (Note that \(x = 0.0\) typically returns -Inf, not NaN).
  • asin(x) and acos(x): Supplying any value where \(|x| > 1.0\). In 16-bit half-float, precision loss in vector normalizations often results in dot(a, b) returning values like \(1.00098\), which immediately causes acos() to return NaN.
  • pow(base, exp): Supplying a negative base when exp is a non-integer, or evaluating \(0.0^0\) (implementation-dependent, often returning NaN).

3. NaN Contamination (Propagation)

IEEE 754 rules dictate that any arithmetic operation involving a NaN input will produce a NaN output:

  • If a single color channel, vector component, or intermediate matrix element turns into NaN, subsequent additions, multiplications, matrix transforms, and dot products consuming that component evaluate to NaN.

Hardware-Specific Deviations and Handling

While the IEEE 754 specification defines strict behavior for Inf and NaN, mobile GPUs and embedded hardware handling GLSL mediump often take implementation-dependent shortcuts:

  • Flushing Subnormals to Zero (FTZ): Many mobile GPUs flush subnormal numbers directly to +0.0. If a divisor underflows to subnormal precision, it may be treated as true zero on the next cycle, triggering an unexpected division-by-zero Inf.
  • Lack of Native NaN Support: Certain older or constrained mobile architectures do not support native NaN or Inf bit patterns and instead clamp calculations to [-65504.0, 65504.0]. On modern mobile architectures (such as ARM Mali, Apple Silicon, and Qualcomm Adreno), full IEEE 754 compliance for half-precision is standard, meaning NaN and Inf behave identically to full-precision floats.