Running JPEG AI Without Dedicated NPU Hardware

Deep learning-based image compression standards like JPEG AI offer superior compression ratios and perceptual quality compared to legacy codecs, but their complex neural architectures raise questions about hardware requirements. While Neural Processing Units (NPUs) provide optimal power efficiency and speed for deep learning inference, modern software and algorithmic optimizations allow JPEG AI models to run on general-purpose hardware such as standard CPUs and GPUs. This article explores whether JPEG AI can achieve practical efficiency without specialized NPUs, examining the technical hurdles, alternative hardware performance, and optimization techniques that make non-NPU execution viable.

The Computational Challenge of JPEG AI

Traditional image formats like JPEG, PNG, or WebP rely on hand-crafted, mathematically simple operations such as the Discrete Cosine Transform (DCT) and simple entropy coding. These algorithms require minimal memory bandwidth and can execute instantaneously on virtually any modern processor.

In contrast, JPEG AI uses deep neural networks (DNNs), typically consisting of convolutional layers, self-attention modules, and learned non-linear transforms. Decoding a single image requires millions, sometimes billions, of floating-point operations (FLOPs). Without acceleration, running these operations sequentially introduces noticeable latency, excessive battery drain on mobile devices, and high CPU utilization.

Running JPEG AI on Modern CPUs

Standard central processing units (CPUs) can execute JPEG AI decoding, but their efficiency depends heavily on instruction set support:

• Vector Extensions: CPUs equipped with modern SIMD (Single Instruction, Multiple Data) extensions—such as ARM Neon, AVX-2, and AVX-512—can parallelize tensor math significantly better than legacy architectures.
• Matrix Acceleration: Modern desktop and mobile processors increasingly include instruction sets specifically designed for matrix multiplication, such as Intel AMX (Advanced Matrix Extensions) and ARM's SME (Scalable Matrix Extension). These features bridge the gap between traditional compute cores and dedicated neural accelerators.
• Latency Bottlenecks: While a modern multi-core CPU can decode a standard JPEG AI image in a few hundred milliseconds, it does so at the cost of high thermal output and energy consumption, making purely CPU-driven decoding sub-optimal for mass image decoding or real-time web browsing.

Utilizing GPUs as an Alternative

For devices lacking an NPU, the graphics processing unit (GPU) serves as the primary fallback:

• Massive Parallelism: GPUs are inherently designed for parallel matrix operations, aligning closely with the mathematical requirements of neural networks.
• API Integration: Using cross-platform compute frameworks such as Vulkan, OpenCL, DirectML, or WebGPU, JPEG AI decoders can offload inference to integrated graphics (iGPUs) found in mainstream laptops and smartphones.
• Real-World Viability: An integrated GPU can achieve decoding latencies comparable to an entry-level NPU. While power consumption is generally higher on an iGPU than on a domain-specific NPU, it provides a practical solution for desktop and portable devices to run JPEG AI smoothly.

Key Optimizations for Non-NPU Hardware

To enable practical efficiency without dedicated AI silicon, JPEG AI implementations rely on several software and architectural optimizations:

1. Quantization: Converting weights and activations from 32-bit floating-point (FP32) to 8-bit integers (INT8) or 16-bit floating-point (FP16) reduces memory footprint by up to 75% and dramatically accelerates throughput on standard CPU vector units.
2. Simplified Entropy Models: The entropy coding stage in learned compression is often an autoregressive bottleneck that executes sequentially. Designing parallel, checkerboard, or non-autoregressive entropy models enables full utilization of multithreaded CPUs and GPUs.
3. Model Pruning and Distillation: Reducing the number of convolutional channels and pruning redundant network parameters decreases the overall FLOP count without severely impacting visual fidelity.

Final Assessment

JPEG AI algorithms can run efficiently without specialized NPU hardware, provided they utilize modern integrated or discrete GPUs alongside optimized inference libraries. On modern CPUs, efficiency is achievable for single-image decodes using SIMD and INT8 quantization, but power-constrained mobile environments still benefit immensely from NPU offloading for sustained, low-power operation.