ALU Recalculation vs Texture Fetching in GLSL

Modern GPU shader optimization often requires developers to choose between recalculating mathematical values dynamically using Arithmetic Logic Units (ALUs) or storing precomputed results to sample via texture fetches. This article breaks down the technical trade-offs between dynamic ALU computation and texture memory operations in GLSL shaders. By evaluating factors such as memory bandwidth saturation, latency hiding, register pressure, and hardware filtering, this guide provides actionable principles for selecting the most performant approach for specific rendering pipelines.

The Shift in Compute-to-Bandwidth Ratios

Historically, texture lookups were preferred over complex mathematical evaluations because early graphics hardware had limited instruction counts and relatively fast access to small textures. Modern GPU architectures have reversed this paradigm. The growth of raw compute power (FLOPS) has vastly outpaced the growth of memory bandwidth.

Today, arithmetic instructions are practically free compared to memory access. A modern GPU can execute dozens or even hundreds of ALU instructions in the time it takes to resolve a single un-cached texture fetch from VRAM. Consequently, the default optimization strategy has shifted toward procedural generation and mathematical recalculation over prebaked lookups.

Advantages of ALU Recalculation

  1. Zero Bandwidth Consumption: Pure mathematical calculations do not consume memory bandwidth, reserving the memory bus for geometry buffers, render targets, and high-frequency textures.
  2. Infinite Analytical Precision: Recalculated values operate at full 32-bit (or 16-bit mediump) floating-point precision, eliminating the banding, compression artifacts, and resolution limits inherent to texture storage.
  3. No Cache Thrashing: Mathematical operations do not evict data from L1 or L2 texture caches, preserving cache capacity for operations that strictly require sampled data.
  4. Predictable Latency: Arithmetic instructions possess fixed, short pipeline latencies that the GPU scheduler can easily hide without stalling warps or wavefronts.

Drawbacks of ALU Recalculation

The primary penalty of intensive ALU usage is register pressure. Evaluating complex analytic functions (such as high-order polynomials, multi-octave noise, or complex ray-surface intersections) increases the number of general-purpose registers (GPRs) required per thread. When register usage per thread exceeds hardware thresholds, the GPU must reduce occupancy—the number of active warps running concurrently on a compute unit. Lower occupancy reduces the GPU's ability to interleave execution and hide whatever memory latency does exist.

Advantages of Texture Memory Fetching

  1. Hardware-Accelerated Filtering: Texture sampling units (TMUs) handle bilinear, trilinear, and anisotropic filtering at the silicon level. Replicating bilinear interpolation purely in ALU instructions requires several additions, multiplications, and fract operations.
  2. O(1) Evaluation of Arbitrary Functions: A texture lookup evaluates non-analytical or computationally prohibitive functions—such as precomputed BRDF lookup tables (LUTs), complex atmospheric scattering profiles, or volumetric absorption—in a single sampling instruction (texture()).
  3. Instruction Count Reduction: Lookups replace long assembly instruction chains with a single fetch instruction, which can reduce shader compilation complexity and code size.

Drawbacks of Texture Fetching

  1. High Memory Latency: A cache miss that requires fetching data directly from VRAM can stall a thread for hundreds of clock cycles. If the GPU cannot switch to another warp due to low occupancy, execution stalls entirely.
  2. Texture Cache Contention and Divergence: Non-coherent UV coordinates (e.g., highly randomized lookups or steep angles relative to the camera) cause frequent L1/L2 cache misses.
  3. Memory Footprint and Bus Saturation: High-resolution LUTs consume VRAM and contribute to bus saturation, creating bottlenecks for memory-bound passes like post-processing or deferred lighting.

Decision Framework: Calculate or Sample?

Choose ALU Recalculation when:

  • The mathematical function can be expressed with a small set of elementary operations (e.g., basic trigonometric approximations, smoothstep curves, color space conversions).
  • The texture lookup would experience poor spatial locality, resulting in high cache miss rates.
  • The shader pass is already memory-bandwidth bound.
  • Higher precision is required than what typical texture formats (such as RGBA8 or RGBA16F) provide.

Choose Texture Fetching when:

  • The calculation requires hardware filtering (e.g., seamless interpolation across a 2D or 3D grid).
  • The algorithm is discontinuous, empirical, or mathematically intractable to solve in real time.
  • The texture access pattern is spatially coherent across adjacent fragments, maximizing L1 cache hit rates.
  • Heavy ALU register usage begins to degrade thread occupancy and overall frame rate.