How Does GLSL Precision Impact Performance and Accuracy?

In OpenGL ES and WebGL shader programming, default precision declarations establish the baseline numerical format for floating-point and integer calculations, creating a direct trade-off between execution speed, memory footprint, and visual fidelity. While choosing a lower precision qualifier can significantly boost arithmetic throughput, reduce register pressure, and lower power consumption on mobile GPUs, it also introduces numerical instability, banding artifacts, and coordinate jitter. Understanding how GPUs process different precision tiers ensures shaders maintain visual correctness without sacrificing hardware performance.

Understanding GLSL Precision Qualifiers

GLSL defines three primary precision qualifiers: lowp, mediump, and highp. These qualifiers dictate the minimum required range and relative precision of floating-point and integer representations:

  • highp (32-bit float): Corresponds to IEEE 754 single-precision floating-point format (24-bit mantissa, 8-bit exponent). It provides the widest numerical range and highest precision, suitable for world-space position calculations, projection matrices, and depth calculations.
  • mediump (16-bit half float): Typically represents FP16 format (10-bit mantissa, 5-bit exponent). It handles values roughly between -65504 and +65504, making it suitable for local texture coordinates, normals, and simple lighting calculations.
  • lowp (8- to 10-bit fixed/float): Provides minimal range (at least -2.0 to +2.0 with 8-bit precision), primarily designed for basic low dynamic range (LDR) color storage and manipulation.

In vertex shaders, the default precision for floats is predefined as highp. In fragment shaders, OpenGL ES specifications require an explicit precision declaration (such as precision mediump float;) because there is no universal default.

Hardware Performance Implications

The performance impact of precision declarations depends heavily on the target hardware architecture:

Mobile and Tile-Based GPUs

Mobile architectures (such as Qualcomm Adreno, ARM Mali, Imagination PowerVR, and Apple silicon) feature dedicated 16-bit execution units alongside standard 32-bit ALUs. When shaders declare mediump, these architectures can execute dual-issue vector instructions or pack two FP16 operations into a single FP32 pipeline stage. This results in:

  • Higher Instruction Throughput: Mobile ALUs often achieve up to twice the arithmetic throughput for mediump calculations compared to highp.
  • Reduced Register Pressure: Lower-precision values consume fewer hardware vector registers. Lower register usage increases thread occupancy per streaming multiprocessor, allowing the GPU to hide memory latency more effectively.
  • Lower Memory Bandwidth and Power Consumption: Shorter data types decrease memory bus traffic and reduce thermal output, preventing GPU throttling on mobile devices.

Desktop GPUs

Most dedicated desktop graphics cards (NVIDIA, AMD, Intel) natively compute all floating-point math at full 32-bit precision (FP32) at the hardware level, regardless of standard GLSL precision qualifiers. On desktop architectures, setting mediump or lowp rarely alters ALU execution speed unless specific modern 16-bit math extensions and driver optimizations are explicitly enabled.

Visual Accuracy and Common Precision Artifacts

Applying an aggressive default precision like mediump across an entire fragment shader can introduce rendering errors due to reduced numerical range and precision limits:

  • Texture Coordinate Aliasing and Jitter: Using mediump for texture UV coordinates over large geometry or animated surfaces leads to noticeable stepping artifacts and texture wobbling, because 16-bit half floats lack sufficient fractional precision at high coordinate values.
  • Color Banding: Smooth gradients and post-processing passes (such as tone mapping, bloom, or ambient occlusion) suffer from quantization steps when calculated in lower precision, producing visible stepped color bands.
  • Specular Highlight Clipping: Specular powers and exponential functions (\(x^n\)) easily exceed the dynamic range limit of mediump (65504.0), causing blown-out flat white areas or arithmetic overflow returning NaN or Infinity.
  • Depth and Distance Calculation Errors: Linear depth buffers, shadow-map comparisons, and raymarching loops lose accuracy rapidly under mediump, causing z-fighting, light leakage, or self-shadowing acne.

Practical Best Practices

To achieve optimal performance without degrading rendering quality, adopt a selective precision strategy:

  • Set precision mediump float; as the Fragment Default: On mobile and WebGL platforms, declaring mediump as the baseline minimizes overhead for basic color blending and standard shading calculations.
  • Override Critical Variables with highp: Explicitly declare variables requiring wide dynamic range or high fractional accuracy as highp. These include screen-space UVs, world-space position reconstructions, depth transformations, and complex lighting exponents.
  • Use lowp Exclusively for Final Output Colors: Restrict lowp to standard 8-bit RGBA color buffers, simple tint calculations, or alpha masking where fractional range beyond \([-2.0, 2.0]\) is unnecessary.
  • Keep Vertex Transformations at highp: Position vectors, normal transformation matrices, and camera transformations must remain at full precision to avoid model deformation and spatial snapping during camera movement.