What Is the JPEG Pleno Standardization Framework?

This article provides an overview of the JPEG Pleno standardization framework developed by the Joint Photographic Experts Group (JPEG). It explores how the standard defines and unifies emerging plenoptic modalities—including light fields, point clouds, and holographic imaging—under a single interoperable ecosystem, detailing its core architecture, functional components, and real-world applications in next-generation imaging.

The Joint Photographic Experts Group defines JPEG Pleno (formally designated as ISO/IEC 21794) as an open standardization framework specifically engineered to capture, represent, compress, and exchange plenoptic visual information. Unlike conventional 2D imaging, which records light intensity and color on a flat grid, plenoptic representations capture the multidimensional behavior of light rays in space. JPEG Pleno addresses this complexity by creating a unified metadata and coding structure that supports advanced viewing experiences, such as depth-dependent refocusing, synthetic perspective shifting, and full 3D scene reconstruction.

The JPEG Pleno framework is structured around three primary visual modalities:

• Light Fields (Part 2): Represents scenes captured via dense camera arrays or microlens sensors. This modality stores multi-view directional information, enabling synthetic depth-of-field manipulation, disparity mapping, and interactive view switching.
• Point Clouds (Part 3): Encodes 3D spatial points with associated attributes such as color, reflectance, and normal vectors. JPEG Pleno provides compression optimized for photorealistic point clouds, serving computer vision, robotics, and immersive media.
• Digital Holography (Part 4): Standardizes interference patterns created by coherent wavefields. This modality enables genuine wavefront reconstruction, delivering continuous parallax and natural eye accommodation without stereoscopic viewing discomfort.

At its structural core, JPEG Pleno establishes a modular architecture comprised of several integrated parts:

1. Framework and Common File Format (Part 1): Defines the overarching system layer, utilizing box-structured file containers (compatible with the ISO Base Media File Format) to house compressed codestreams, color profiles, spatial relationships, and metadata.
2. Modality-Specific Codecs (Parts 2–4): Implements compression schemes tailored to the high correlations present within each specific modality, ensuring high coding efficiency without sacrificing visual fidelity.
3. Conformance and Reference Software (Parts 5 and 6): Provides test suites, reference implementations, and verification procedures to ensure interoperability across diverse hardware and software ecosystems.

JPEG Pleno also incorporates advanced functional features designed for practical deployment:

• Scalability: Allows codestreams to be decoded at various spatial, quality, and view resolutions to accommodate different network bandwidths and processing capabilities.
• Random Access: Enables decoders to fetch and unpack only the specific viewpoints or spatial regions required by a user at a given moment, saving processing power and memory.
• Privacy and Security: Embeds native support for encryption, authentication, and digital rights management within the file hierarchy.

By unifying light fields, point clouds, and holography under one standardized architecture, the Joint Photographic Experts Group positions JPEG Pleno as the foundational standard for immersive visual media, supporting applications across augmented and virtual reality, medical imaging, cultural preservation, and geographic information systems.