VRM Metadata for Facial Expressions and Eye Tracking

VRM extends standard glTF 2.0 to transform generic 3D models into interoperable, humanoid avatars. While glTF provides the baseline geometry, skeleton, and morph targets, it lacks semantic context for humanoid behavior. VRM bridges this gap by embedding specialized metadata—primarily through the VRMC_vrm extension in VRM 1.0 (or the VRM extension in VRM 0.x)—that standardizes facial expression mappings, gaze control, and eye movement across disparate applications.

Standardized Facial Expressions (expressions)

Standard glTF supports morph targets (blend shapes), but it references them solely as indexed numeric targets without semantic meaning. VRM introduces an expressions schema (formerly blendShapeMaster in VRM 0.x) that binds these low-level morph targets to universal expression presets.

The metadata schema defines three categories of expressions:

• Emotional Presets: Universal states such as happy, angry, sad, relaxed, and surprised.
• Viseme/Phoneme Presets: Lip-sync targets for speech animation, specifically aa, ih, ou, ee, and oh.
• Procedural/Utility Presets: Eyelid controls including blink, blinkLeft, and blinkRight, as well as directional gaze targets (lookUp, lookDown, lookLeft, lookRight).
• Custom Presets: Developer-defined expressions that do not fit standard categories.

Within the metadata, each expression definition contains explicit instructions on how the avatar's state changes when the expression weight shifts from 0.0 to 1.0:

• Morph Target Binds: A list detailing target mesh node indices, morph target indices, and their corresponding weight multipliers. A single semantic expression can trigger multiple morph targets across separate meshes simultaneously.
• Material Color Binds: Metadata to drive material property changes—such as tint colors, UV offsets, or emission values—enabling effects like blushing or fading eyes alongside geometry deformation.
• Override Settings: Rules defining how competing blend shapes interact, such as automatically zeroing out standard eye blinks when a specific emotional expression is active (overrideBlink, overrideLookAt, overrideMouth).

Gaze and Eye Tracking Metadata (lookAt)

Standard glTF has no built-in concept of an eye gaze target or gaze orientation. VRM solves this by providing the lookAt metadata object, which allows applications to direct an avatar's line of sight toward arbitrary 3D coordinates.

The lookAt configuration includes:

• Reference Point (offsetFromHeadBone): A three-dimensional vector defining the exact origin point between the eyes relative to the head bone. This ensures eye-tracking systems calculate the line-of-sight vector accurately regardless of model scale or head geometry.
• Look-At Type (type): Defines the mechanical approach used to move the eyes:
  • bone: The gaze is driven by rotating dedicated skeletal joints (leftEye and rightEye bones defined in the VRM humanoid bone mapping).
  • expression: The gaze is driven by triggering directional morph targets (lookUp, lookDown, lookLeft, lookRight).
• Range Mapping Curves: Mathematical parameters that dictate how angular displacement (yaw and pitch in degrees toward a target) translates to either bone rotation angles or morph target weights. The metadata specifies separate curves for:
  • Horizontal inner movement (moving toward the nose).
  • Horizontal outer movement (moving away from the nose).
  • Vertical upward movement.
  • Vertical downward movement.

By standardizing morph target bindings and gaze trajectory curves in machine-readable JSON inside the glTF container, VRM ensures that facial animations, lip-sync algorithms, and hardware-based eye trackers function predictably across any supporting virtual engine.