What Is Integer Quantization? INT8 and INT4 Guide

Integer quantization is a model optimization technique that compresses deep learning neural networks by converting high-precision floating-point weights and activations into lower-bit integer representations, such as 8-bit (INT8) or 4-bit (INT4). This article covers how integer quantization functions, the underlying binary mechanics of weight compression, the mathematical mapping required to preserve precision, and the trade-offs between computational performance and model accuracy.

The Standard Precision Problem in Deep Learning

Deep learning models are traditionally trained using 32-bit single-precision floating-point values (FP32). In the IEEE 754 standard, each FP32 number consumes 32 bits (4 bytes) of memory, structured as:

• 1 sign bit: Determines positive or negative.
• 8 exponent bits: Defines the magnitude range.
• 23 mantissa (fraction) bits: Determines the precision.

While FP32 offers high dynamic range and numerical stability during training, it creates substantial bottlenecks during deployment. Large Language Models (LLMs) and massive vision models with billions of parameters require tens or hundreds of gigabytes of VRAM just to store weights, limiting their ability to run efficiently on edge devices or standard enterprise hardware.

How Quantization Compresses Binary Weights

Integer quantization replaces 32-bit floating-point weights with compact integer formats. This process directly reduces the number of bits allocated to each parameter in the binary number system:

1. INT8 Quantization (4x Compression)

INT8 utilizes an 8-bit signed integer format capable of representing \(2^8 = 256\) distinct values (ranging from -128 to 127). By reducing the bit-width from 32 bits to 8 bits, memory consumption per weight is reduced by a factor of 4. A 7-billion-parameter FP32 model requiring roughly 28 GB of storage is compressed down to approximately 7 GB.

2. INT4 Quantization (8x Compression)

INT4 reduces the precision further to a 4-bit integer, representing \(2^4 = 16\) distinct values (typically ranging from -8 to 7, or 0 to 15 if unsigned). Two 4-bit values can be packed into a single byte of memory. This achieves an 8x reduction in storage and memory bandwidth compared to FP32, allowing a 7-billion-parameter model to fit into under 4 GB of VRAM.

The Mathematical Mapping Process

To compress continuous floating-point weights into a discrete, limited set of integer values, a linear transformation is applied using a scale factor (\(S\)) and a zero-point offset (\(Z\)).

The standard affine quantization formula is:

\[q = \text{round}\left(\frac{x}{S}\right) + Z\]

Where: * \(x\) is the original floating-point weight or activation. * \(S\) (Scale) is a positive floating-point value that defines the step size between consecutive integer values: \(S = \frac{x_{\max} - x_{\min}}{q_{\max} - q_{\min}}\). * \(Z\) (Zero-point) is an integer that maps the real zero to the quantized domain, ensuring that exact zero in floating-point maps without error to a representable integer. * \(q\) is the resulting quantized integer.

To reconstruct the values during computation (dequantization), the inverse formula is used:

\[\tilde{x} = S \cdot (q - Z)\]

In symmetric quantization, the zero-point \(Z\) is set to 0, which simplifies hardware arithmetic at the cost of a slightly reduced dynamic range for asymmetric distributions.

Hardware and Performance Advantages

Compressing weights into INT8 and INT4 formats provides major system-level advantages beyond raw disk storage:

• Reduced Memory Bandwidth: Deep learning inference is frequently memory-bandwidth bound. Transferring 8-bit or 4-bit values from memory to compute units requires significantly less time and energy than transferring 32-bit floats.
• Faster Integer Arithmetic: Modern CPUs and GPUs (such as NVIDIA Tensor Cores) contain specialized vector units that execute integer matrix multiplications (INT8/INT4 GEMM) at multiple times the throughput of floating-point operations.
• Lower Power Consumption: Fixed-point integer arithmetic logic units (ALUs) consume significantly less electrical power and silicon area compared to complex floating-point units (FPUs).

Methods of Implementation

Quantization is typically applied using one of two strategies:

• Post-Training Quantization (PTQ): Weights are quantized after the model is fully trained. This requires minimal computation and no retraining, making it fast and easy to apply, though INT4 PTQ may cause noticeable degradation without calibration.
• Quantization-Aware Training (QAT): The network models the effects of low-precision rounding during the training or fine-tuning phase using “fake quantization” nodes. This allows the backpropagation algorithm to adjust remaining weights to compensate for precision loss, yielding higher accuracy at low bit-widths like INT4.