glTF Stereoscopic Camera Rigs for VR Rendering

This article explains how the glTF (GL Transmission Format) 2.0 specification enables stereoscopic camera rigs for virtual reality (VR) rendering. While glTF natively standardizes monoscopic perspective cameras, it accommodates dual-camera stereoscopic setups through scene graph node hierarchies, spatial transform offsets, and runtime integration with XR frameworks. Readers will learn the structural design of stereoscopic rigs in glTF, how projection parameters and interpupillary distance are represented, and how VR engines interpret these assets.

The glTF Core Camera Model

The base glTF 2.0 specification defines cameras as reusable resources referenced by nodes within a scene graph. The specification supports two camera types: perspective and orthographic. For immersive and VR applications, perspective cameras are used, defined by properties such as:

  • yfov: The vertical field of view in radians.
  • znear: The distance to the near clipping plane.
  • zfar: The distance to the far clipping plane (optional, allowing for infinite projections).
  • aspectRatio: The aspect ratio of the field of view.

Core glTF cameras are inherently monoscopic and assume a symmetrical viewing frustum centered along the camera's local negative Z-axis. Because the standard does not include a dedicated "stereoscopic camera" type, stereo capabilities are built using the format's structural composition.

Constructing Stereo Rigs via Node Hierarchies

Stereoscopic rendering requires two distinct viewpoints separated by an interpupillary distance (IPD), typically averaging around 63 to 65 millimeters. In glTF, this is achieved by composing a node hierarchy representing a camera rig:

  1. Parent Rig Node: A root node representing the user's head or tracking origin. Transformations applied to this node move and rotate the entire rig within the virtual environment.
  2. Left Eye Node: A child node offset along the local X-axis in the negative direction (e.g., -0.0315 meters for an average IPD). This node references a perspective camera object.
  3. Right Eye Node: A child node offset along the local X-axis in the positive direction (e.g., +0.0315 meters). This node references an identical or mirrored perspective camera object.
{
  "nodes": [
    {
      "name": "VR_Camera_Rig",
      "children": [1, 2]
    },
    {
      "name": "Camera_LeftEye",
      "translation": [-0.032, 0.0, 0.0],
      "camera": 0
    },
    {
      "name": "Camera_RightEye",
      "translation": [0.032, 0.0, 0.0],
      "camera": 0
    }
  ],
  "cameras": [
    {
      "type": "perspective",
      "perspective": {
        "yfov": 1.5708,
        "znear": 0.05
      }
    }
  ]
}

By referencing the same camera definition index (camera: 0), both eyes share common projection characteristics such as clipping planes and vertical field of view, while their distinct node transforms define the stereoscopic baseline.

Addressing Asymmetric Frustums and Runtime Projection

Head-mounted displays (HMDs) typically require off-axis, asymmetric projection matrices because the human eye and physical display lenses are rarely centered directly in front of each display panel.

Core glTF perspective definitions only support symmetric frustums defined by a single vertical field of view and aspect ratio. To resolve this limitation in VR workflows, glTF files establish the nominal design parameters and spatial hierarchy, while the runtime engine (such as WebXR, OpenXR, Unity, or Unreal Engine) dynamically overrides the camera projection matrices at render time using real-time tracking data and lens distortion profiles provided by the hardware device.

Semantic Labeling with Metadata and Extensions

To allow ingestion pipelines to automatically identify stereo pairs, developers utilize glTF metadata capabilities:

  • extras Objects: Custom application-level metadata can be attached directly to the left and right nodes (e.g., {"extras": {"eye": "left"}}). This signals to loader scripts which viewport or render target each camera feeds.
  • Naming Conventions: Standardized node names (such as eye_left, eye_right, or specific OpenXR node names) are frequently used by loaders to map glTF nodes to VR headset render loops automatically.
  • Vendor Extensions: Proprietary and vendor-specific glTF extensions can define explicit stereoscopic parameters, custom IPD behaviors, convergence angles, or non-symmetric projection bounds when the asset must dictate strict rendering behavior without hardware intervention.

Integration with WebXR and Native VR Engines

When loaded into a 3D engine, the glTF hierarchy functions as an anchor for the hardware-driven VR camera system:

  • Transform Binding: The parent rig node's world transform is mapped to the tracking space origin.
  • Viewport Assignment: The left and right glTF child nodes are assigned to the respective left and right viewports or render passes of the display pipeline.
  • Animation and Kinematics: Because glTF supports skeletal animation and transform keyframing, predefined stereo camera paths (such as cinematic fly-throughs or roller-coaster rides) can animate the parent rig node while maintaining accurate stereoscopic separation between the child eye nodes throughout the sequence.