Embedding Interactive Hotspots Inside glTF Nodes

Embedding interactive hotspots or annotation pins into glTF (GL Transmission Format) scenes bridges the gap between raw 3D assets and interactive user interfaces. Because the core glTF 2.0 specification focuses primarily on efficient runtime delivery of geometry, materials, and animations, incorporating interaction requires utilizing structural nodes, custom metadata, or rendering-engine capabilities. This article details the primary mechanisms used to position, define, and render interactive pins relative to glTF nodes.

1. Empty Locator Nodes (Transform Anchors)

The most standard-compliant and software-agnostic approach is using empty nodes—often referred to as locators, null objects, or dummies in 3D modeling software like Blender, Maya, or 3ds Max.

  • Hierarchical Placement: An empty node is inserted into the glTF scene graph as a child of the specific mesh or bone it relates to. This ensures that if the parent object moves, rotates, or animates, the hotspot's coordinate system automatically inherits those transformations.
  • Transform Extraction: The rendering engine (such as Three.js, Babylon.js, or PlayCanvas) traverses the glTF nodes array, identifies these locator nodes by name (e.g., HOTSPOT_Engine_Core), and retrieves their world transformation matrices.
  • Screen-Space Projection: At runtime, the 3D position vector derived from the node's local matrix is projected onto the 2D viewport using the active camera's view-projection matrix, allowing standard HTML/CSS elements to track the 3D position dynamically.

2. glTF extras Metadata Property

Every major glTF object—including nodes, meshes, and scenes—supports an optional extras property reserved for application-specific metadata.

  • Embedded Payloads: Developers can store arbitrary JSON objects directly inside a node’s definition. For an annotation pin, this might include titles, textual descriptions, URLs, or styling states:

    {
  "name": "Pin_Sensor_A",
  "translation": [0.0, 1.5, -0.5],
  "extras": {
    "type": "annotation",
    "title": "Temperature Sensor",
    "description": "Monitors intake manifold temperatures.",
    "cameraTarget": [0.0, 1.5, 0.0]
  }
}

  • Runtime Parsing: When the loader processes the glTF file, client-side scripts parse the extras field, instantiate corresponding UI components, and attach click or hover listeners to the coordinates.

3. Dedicated glTF Extensions

For enterprise pipelines requiring standardized schemas, custom or vendor glTF extensions provide formal validation.

  • Vendor Extensions: Teams can declare custom extensions in the asset header via extensionsUsed and extensionsRequired (e.g., EXT_interactive_annotations).
  • Schema Definition: An extension can strictly define schemas for annotation behavior, including trigger radii, audio attachments, state changes, or target camera perspectives for guided tours.

4. Geometric Marker Meshes and Raycasting

Another mechanism involves storing the physical representation of the pin directly inside the glTF asset as real geometry.

  • Visual Anchors: Pins are modeled as 3D geometry (such as spheres, icons, or directional cones) and assigned designated materials or names within the node hierarchy.
  • Raycasting: Modern web rendering engines use raycasting algorithms to detect pointer intersections with these specific mesh nodes. When an intersection occurs, the application triggers modal dialogues, camera transitions, or custom events.
  • Billboarding: To keep 2D-style pins readable regardless of camera orientation, shaders or update loops apply billboarding techniques to the marker node, forcing its local rotation to continually face the camera.

5. Web Component Integration (Surface and Normal Snapping)

High-level 3D web components, such as Google's <model-viewer>, provide built-in mechanisms that translate glTF surface points directly into interactive slots.

  • Surface Coordinates: Hotspots are assigned coordinates using surface-normal projections (e.g., data-position and data-normal attributes).
  • Occlusion Calculation: The component checks the depth buffer against the glTF node hierarchy to dynamically hide or fade annotation pins when solid parts of the 3D model occlude the node's position.