3D Gaussian Splatting glTF Extensions Explained

This article explores the ongoing efforts and experimental extensions aimed at integrating 3D Gaussian Splatting (3DGS) data into the glTF (GL Transmission Format) ecosystem. While 3DGS traditionally relies on Stanford PLY files for storage and transmission, the Khronos Group and the wider open-source community are actively developing extensions—most notably the draft KHR_gaussian_splatting proposal and custom attribute-based schemas—to bring spatial hierarchy, compression, and universal rendering capabilities to volumetric radiance data.

The Problem with Current 3DGS Delivery

The standard exchange format for 3D Gaussian Splatting has predominantly been the legacy .ply format. While easy to dump from machine learning training pipelines, PLY files are heavy, lack built-in compression, and do not natively support scene-graph hierarchies, coordinate transformations, materials, or lighting environments.

Integrating 3DGS into glTF 2.0 solves these issues by enabling developers to treat splat clouds as first-class scene elements alongside conventional polygonal meshes, cameras, and animations.

The Proposed KHR_gaussian_splatting Extension

The primary initiative is an experimental Khronos Group extension proposal typically referred to as KHR_gaussian_splatting. This official draft aims to define standard conventions for encoding 3D Gaussian primitives directly inside the glTF JSON structure and binary buffer payloads.

Under this specification, splat parameters are mapped directly to binary accessors:

  • Centers (Positions): Stored using the standard glTF POSITION semantic.
  • Rotations: Stored as normalized quaternions using custom or extended attributes.
  • Scales: Stored as 3D vectors representing the anisotropic stretch along the principal axes.
  • Opacity/Alpha: Encoded linearly or via logit space into scalar accessors.
  • Spherical Harmonics (SH): Handled either as diffuse-only coefficients (degree 0) or higher-order directional coefficients (degrees 1–3), segmented into distinct accessor arrays to enable progressive level-of-detail loading.

Community and Vendor Approaches

Prior to broad ratification of a single unified extension, various engine developers and platforms have experimented with distinct packaging techniques:

1. Custom Attribute Multiplexing (_FEATURE Semantic)

Early implementations—popularized in WebGL and WebGPU engines like PlayCanvas, Babylon.js, and Three.js—store Gaussian parameters using glTF's user-defined attribute system. By prefixing attributes with an underscore (such as _ROTATION, _SCALE, and _EXTRA_SH), runtimes can parse raw point clouds containing splat parameters without breaking compatibility with engines that ignore unrecognized attributes.

2. Texture-Encoded Splat Data

Some experimental pipelines serialize Gaussian properties into 2D data textures instead of point-cloud vertex buffers. By arranging position, covariance, and color data into image channels (often utilizing the KHR_texture_basisu extension for GPU-friendly texture compression), engines can sample splats in compute or fragment shaders. This approach benefits directly from standard image streaming and mipmapping infrastructures.

3. 3D Tiles Integration (3DTILES_content_gltf)

The Cesium community has introduced methods to stream massive, geospatial 3D Gaussian splat sets by encapsulating them within glTF assets served through the 3D Tiles 1.1 standard. This method relies on bounding volume hierarchies to load, cull, and unload partitioned glTF chunks containing splat primitives depending on the viewer's camera position.

Compression Strategies Within glTF

Wrapping 3DGS in glTF allows experimental pipelines to leverage existing glTF compression extensions:

  • EXT_meshopt_compression: Applied to point-cloud accessors to quantize quaternions, scale vectors, and color harmonics, drastically reducing transfer size with minimal decode overhead on the web.
  • KHR_draco_mesh_compression: Used in experimental setups to compress point sets, though its decompression cost on the CPU can sometimes create a bottleneck for high-splat-count scenes.

These experimental extensions represent a critical transition for 3D Gaussian Splatting, shifting it from an isolated research output into a standardized, web-ready asset format for real-time 3D graphics.