Why BC1 Causes Normal Map Artifacts in DirectX

Compressing normal maps using the standard BC1 (DXT1) format in DirectX consistently produces severe visual artifacts, including harsh faceting, blocky banding, and distorted specular highlights. This article explains the technical reasons behind these decompression artifacts, detailing how the 5:6:5 color quantization, shared single-line endpoint interpolation, cross-channel correlation assumptions, and subsequent vector normalization in shaders combine to degrade normal data.

The Limitation of 16-Bit Endpoints and 2-Bit Indices

BC1 divides textures into 4x4 pixel blocks and evaluates color values by establishing two representative 16-bit RGB endpoints per block. These endpoints use 5 bits for Red, 6 bits for Green, and 5 bits for Blue. The intermediate pixels inside the 4x4 block are then assigned 2-bit indices that interpolate along a straight line between the two endpoints, offering only four distinct color steps.

Because normal vectors require high numerical precision to represent smooth angular gradations, 5-bit and 6-bit quantization introduces severe precision loss. The 2-bit index palette fails to represent subtle curvature, flattening smooth surfaces into a stepped, blocky staircase pattern across each 4x4 region.

The Assumption of Linear RGB Correlation

BC1 was designed for standard diffuse color data, where luminance and chrominance across a 4x4 pixel block typically correlate closely along a single line in RGB space.

Normal maps store spatial vectors \((X, Y, Z)\) mapped into the range \([0, 1]\) via the formula: \[\text{Color} = 0.5 \times \text{Vector} + 0.5\]

Directional vectors across a surface change continuously along a spherical manifold rather than a straight line in 3D color space. Forcing independent \(X\) and \(Y\) directional derivatives onto a single 1D interpolation axis forces compromises: either the X component or the Y component must sacrifice precision, introducing severe cross-channel bleeding and directional skewing.

Amplification via Vector Normalization

Normal vectors are unit vectors, defined by the constraint \(X^2 + Y^2 + Z^2 = 1\). When BC1 decompresses a block, the resulting \((X, Y, Z)\) coordinates rarely form a unit vector due to independent channel rounding and interpolation errors.

Pixel shaders typically renormalize the sampled vector: \[\vec{N} = \text{normalize}(\text{sample})\]

Renormalization drastically magnifies subtle quantization errors. Small planar deviations in the quantized \(X\) and \(Y\) channels cause significant shifts in the final vector's angle. Because specular lighting models (such as Blinn-Phong or Cook-Torrance) depend heavily on the dot product between the surface normal and the half-vector raised to high specular powers, minute angular errors appear as glaring jagged edges and specular noise on rendered models.

Modern Solutions

Because of these structural flaws in BC1, modern DirectX pipelines avoid BC1 for normal maps entirely:

• BC5 (DXGI_FORMAT_BC5_UNORM): The industry standard for tangent-space normal maps. It provides two separate 8-bit endpoint channels for \(X\) and \(Y\), while \(Z\) is calculated dynamically in the pixel shader via \(Z = \sqrt{1 - X^2 - Y^2}\).
• BC7 (DXGI_FORMAT_BC7_UNORM): A high-quality alternative that supports multiple partitioning lines and higher bit precision per block, suitable when all three vector components or additional alpha masks must be retained.