Modifying JPEG for Hyperspectral and Multispectral Data

Standard JPEG was engineered for three-channel, 8-bit natural color imagery, making it fundamentally incompatible with the rich dimensional and radiometric requirements of multispectral and hyperspectral data. To compress these datasets effectively, the baseline JPEG framework is modified to support higher bit depths, inter-band spectral decorrelation, three-dimensional transform coding, and scalable codestream architectures. This article examines the specific structural and mathematical adaptations required to apply JPEG-based compression pipelines to multi-band scientific imagery.

Dynamic Range and Bit Depth Expansion

Standard baseline JPEG relies on an 8-bit dynamic range per channel (0–255 values). Multispectral and hyperspectral sensors typically record data at 10, 12, 14, or 16 bits per sample to preserve radiometric accuracy and subtle physical reflectance variations.

Applying JPEG to these datasets requires shifting from baseline DCT-based profiles to the JPEG extended sequential or lossless modes, which support up to 12-bit precision, or rewriting the arithmetic and quantization units to handle 16-bit integer and floating-point inputs. Without this modification, scientific data must undergo lossy scaling or requantization before compression, resulting in unrecoverable dynamic range truncation.

Spectral Decorrelation

Standard JPEG applies a color space conversion (typically RGB to YCbCr) to decorrelate three visible color channels. Hyperspectral images, however, contain dozens to hundreds of narrow, contiguous spectral bands exhibiting extreme cross-channel redundancy.

To address this, modern JPEG-based pipelines incorporate a dedicated spectral decorrelation transform prior to spatial compression:

• Karhunen-Loève Transform (KLT) / Principal Component Analysis (PCA): The statistically optimal linear transform for maximizing energy compaction across bands. It concentrates the majority of spectral information into a few high-variance components, allowing subsequent bands to be aggressively quantized or discarded.
• 1D Discrete Cosine Transform (DCT) or Discrete Wavelet Transform (DWT): Applied down the spectral axis as a lower-complexity alternative to KLT that avoids the computational overhead of computing covariance matrices.
• Inter-Band Linear Prediction: Utilizing modified differential pulse-code modulation (DPCM) schemes to predict the values of a given band based on its immediate spectral neighbors, leaving only residual errors for DCT encoding.

From 2D Spatial Blocks to 3D Transform Coding

Baseline JPEG segments single-plane 2D images into \(8 \times 8\) pixel blocks and applies a two-dimensional Discrete Cosine Transform. For hyperspectral compression, this algorithm is expanded into a 3D-DCT acting on volumetric prisms (typically \(8 \times 8 \times 8\) or \(8 \times 8 \times 16\) spatial-spectral voxels).

This 3D transform maps spatial and spectral frequencies simultaneously into a three-dimensional frequency matrix. The traditional 2D "zigzag" scanning pattern used to group zero and non-zero coefficients is replaced with a 3D spherical, pyramidal, or cylindrical scan path optimized for volumetric energy distribution.

Band-Adaptive Quantization and Bit Allocation

In conventional JPEG, quantization matrices are designed based on the human visual system's contrast sensitivity function. Hyperspectral and multispectral images are primarily analyzed by machine algorithms (such as classification, target detection, and spectral unmixing) rather than human eyes.

Modifications to the quantization stage include:

• Analytical Quantization Matrices: Quantization tables derived from spectral signal-to-noise ratios (SNR) and the relative radiometric variance of individual bands rather than human visual perception.
• Band-Specific Weighting: High-priority absorption features or atmospheric windows are assigned finer quantization steps, while water-absorption bands contaminated by noise are quantized aggressively to preserve transmission bandwidth.

Formalized Implementations: JPEG 2000 Part 2

While classic DCT JPEG can be modified experimentally, these requirements led the Joint Photographic Experts Group to formalize these extensions within JPEG 2000 Part 2 (Extensions, ISO/IEC 15444-2).

JPEG 2000 Part 2 natively supports multi-component transformations, allowing arbitrary spectral decorrelation matrices (such as custom KLT or multi-level DWT along the spectral axis) coupled with 2D spatial wavelet transforms. This architecture provides mathematically lossless or mathematically controlled lossy compression, native support for up to 38-bit precision, and arbitrary numbers of spectral bands within a standardized container format (JPX).