Declaring u8vec4 in GLSL for Weight Unpacking

In GPU-accelerated machine learning inference, optimizing memory bandwidth by quantizing neural network weights to 8-bit integers is standard practice. This article explains how to declare and use the u8vec4 type in GLSL using explicit arithmetic extensions, allowing shaders to unpack four quantized 8-bit unsigned weights simultaneously into floating-point vectors for tensor operations.

Enabling 8-Bit Integer Extensions

Standard GLSL lacks native 8-bit scalar and vector types by default. To declare and manipulate a u8vec4 variable, you must enable the explicit arithmetic types extension at the top of your shader:

#version 450
#extension GL_EXT_shader_explicit_arithmetic_types_int8 : require

Declaring u8vec4 Variables

Once the extension is enabled, u8vec4 becomes a first-class type representing a vector of four 8-bit unsigned integers. You can declare u8vec4 variables as local variables, function parameters, or members within buffer storage blocks.

Local Declaration and Initialization

// Uninitialized local declaration
u8vec4 quantizedWeights;

// Explicit initialization with 8-bit literals (using the 'u8' suffix)
u8vec4 biasVector = u8vec4(128u8, 64u8, 255u8, 0u8);

Buffer Storage Block Declaration

Quantized neural network weights are typically stored in uniform or storage buffers (SSBO). You can map incoming raw data directly to u8vec4:

layout(std430, binding = 0) readonly buffer WeightBuffer {
    u8vec4 weights[];
};

Unpacking and Dequantization Example

After loading a u8vec4, the integer values must be dequantized back into standard 32-bit floats (vec4) using a scale factor and zero-point offset before performing dot products or matrix multiplications:

// Fetch 4 packed weights
u8vec4 rawWeights = weights[gl_GlobalInvocationID.x];

// Parameters for affine dequantization: real_value = (quantized_value - zero_point) * scale
float scale = 0.025;
float zeroPoint = 128.0;

// Cast to high-precision float vector and apply dequantization
vec4 unpackedWeights = (vec4(rawWeights) - zeroPoint) * scale;

Fallback: Unpacking Without Explicit 8-Bit Extensions

If the target hardware or driver does not support GL_EXT_shader_explicit_arithmetic_types_int8, standard GLSL can unpack a 32-bit unsigned integer (uint) into four normalized floats using native unpack functions:

uint packedData = rawBuffer[index];

// Automatically unpacks a uint into four normalized [0.0, 1.0] floats
vec4 normalizedWeights = unpackUnorm4x8(packedData);