Bake Ambient Occlusion into glTF Vertex Colors

Baking ambient occlusion directly into vertex colors is an efficient optimization technique that enhances depth perception while eliminating the memory overhead and draw calls of dedicated ambient occlusion texture maps. In glTF 2.0 models, this process relies on computing surface shading per vertex and mapping the resulting occlusion values to the mesh's primary color attribute. This guide outlines the essential tools, processing workflows, and specification requirements for embedding vertex-level ambient occlusion into glTF assets.

How glTF Handles Vertex Colors

The glTF 2.0 specification natively supports vertex colors through the COLOR_0 attribute on mesh primitives. By default in the standard PBR metallic-roughness workflow, the values in COLOR_0 act as a linear multiplier for the material's baseColorFactor and baseColorTexture.

Because glTF does not define a dedicated vertex-level occlusion channel, baking ambient occlusion into COLOR_0 works by storing the grayscale occlusion value directly in the RGB channels. When rendered, this darkens crevices and contact points without requiring an occlusionTexture.

Baking via DCC Software (Blender Workflow)

Digital content creation tools provide built-in solutions for baking occlusion directly to mesh geometry:

  1. Dirty Vertex Colors (Approximation): In Blender's Vertex Paint mode, selecting Paint > Dirty Vertex Colors calculates an ambient occlusion approximation based on surface curvature and angle. This method is fast and requires no ray tracing, though it works best on high-density meshes.
  2. Ray-Traced Cycles Bake: For higher accuracy, assign an Ambient Occlusion input node to a material, create a new Color Attribute (set to Color and Byte Color or Float Color), and use Blender's bake panel set to bake the material to the active Color Attribute.
  3. glTF Export: When exporting to .gltf or .glb, ensure that Data > Mesh > Vertex Colors (or Attributes) is checked. The exporter will package the active color layer as the COLOR_0 accessor.

Baking via Mesh Processing Tools and CLI

Automated pipelines and programmatic pipelines offer scalable ways to inject ambient occlusion without manual 3D suite workflows:

  • MeshLab: The Filters > Color Creation and Processing > Ambient Occlusion per Vertex filter casts rays over a hemisphere from each vertex to compute exact line-of-sight occlusion, storing the result directly as vertex colors.
  • Houdini: Using an Attribute VOP or an Occlusion node, occlusion values can be computed per-point and mapped to the standard Cd (diffuse color) attribute before passing through the glTF ROP exporter.
  • Custom Ray Tracing Scripts: Using libraries such as Trimesh or Embree in Python/C++, you can shoot hemisphere rays from vertex positions along vertex normals to calculate accessibility factors (0.0 to 1.0) and write them directly into a glTF binary buffer mapped to COLOR_0.

Best Practices for Density and Quantization

  • Mesh Density: Vertex-baked ambient occlusion depends entirely on polygon density. Low-poly flat surfaces cannot capture fine contact shadows; the technique is most effective on stylized assets, organic shapes, or geometry with uniformly distributed vertices.
  • Data Types and Compression: To keep file sizes small, encode COLOR_0 as UNSIGNED_BYTE normalized integers (5121) rather than 32-bit floats.
  • Color Space Management: Ambient occlusion is linear data. Ensure baking tools write linear RGB values to avoid double-gamma correction when glTF engines interpret COLOR_0 according to the sRGB/Linear standards defined by the glTF specification.