How glTF Plans to Support 3D Gaussian Splatting

The rapid emergence of 3D Gaussian Splatting and volumetric neural representations has introduced a paradigm shift in real-time 3D capture and rendering, exposing the limitations of traditional polygonal pipelines. To meet this demand, the Khronos Group and the broader glTF ecosystem are actively developing standardized approaches to incorporate Gaussian Splats into the glTF format. This initiative focuses on defining standard splat attributes, enabling effective compression, and integrating volumetric point data directly alongside traditional polygonal meshes within a unified scene graph.

Moving Beyond the PLY Format

Early implementations of 3D Gaussian Splatting relied heavily on Stanford .ply files to store point cloud data. While functional for academic experimentation, the PLY format lacks standard conventions for scene hierarchy, coordinate systems, material interactions, animation, and web-ready compression.

The glTF ecosystem addresses these shortcomings by treating Gaussian Splats as a first-class citizen of the modern 3D transmission pipeline, turning raw point clouds into fully functional, distributable 3D assets.

The Extension-Based Architecture

Rather than overhauling the core specification, the Khronos Group is leveraging glTF's modular extension framework. The proposed support centers on dedicated vendor and Khronos (KHR_) extensions designed specifically for Gaussian Splatting.

Under this framework, a Gaussian Splat is defined using primitive attributes that extend glTF's standard buffer definitions:

  • Position: Standard VEC3 Cartesian coordinates for the splat centroid.
  • Scale and Rotation: VEC3 vectors for non-uniform scaling combined with VEC4 quaternions representing orientation to construct the 3D covariance matrix.
  • Opacity: Scalar alpha values defining splat transparency.
  • Spherical Harmonics: Multi-degree spherical harmonic coefficients that describe view-dependent color shifts.

Texture-Based vs. Buffer-Based Storage

The glTF working groups and community implementers have explored two primary methods for structuring Gaussian data:

  1. Point Cloud Primitives: Storing splats directly as vertices inside glTF binary buffers (.bin), where each vertex represents a Gaussian ellipsoid. This method integrates cleanly with existing glTF buffer accessors.
  2. Texture-Based Packing: Encoding Gaussian attributes into 2D texture arrays (data textures). This approach allows engines to leverage standard hardware interpolation and texture pipelines, often simplifying WebGL and WebGPU shader implementations.

Compression and Transmission Pipelines

Uncompressed Gaussian Splats generate file sizes that frequently exceed hundreds of megabytes, making them impractical for web delivery and real-time streaming. The glTF ecosystem plans to mitigate this through advanced compression standards:

  • Attribute Quantization: Truncating or packing high-degree spherical harmonics and normalizing scale/rotation values to lower-bit precisions (such as 8-bit or 16-bit integers).
  • Universal Texture Compression: Utilizing KHR_texture_basisu (KTX 2.0 with Basis Universal) when splats are stored as data textures, drastically reducing payload sizes over the network.
  • Geometry Compression: Adapting Draco or Meshopt compression pipelines to handle the specific attribute distributions of dense point-based splats.

Hybrid Scene Graph Integration

A central advantage of bringing Gaussian Splats into glTF is hybrid rendering. By utilizing glTF’s node hierarchy, developers can attach splat data to standard scene nodes. This allows neural reconstructions of real-world environments to interact with traditional rasterized assets, including rigid-body transforms, camera paths, skeletal animations, and directional lighting, all within a single file.