How iOS Quick Look Converts glTF to USDZ On-the-Fly

Apple’s AR Quick Look enables users to preview 3D models seamlessly across iOS using Safari, Messages, and Files. While Apple natively built Quick Look around Pixar’s Universal Scene Description (USDZ) ecosystem, modern iOS updates have integrated pathways to handle the web-standard glTF (GL Transmission Format) and its binary variant, GLB. This article breaks down the internal architecture, material translation, and runtime pipelines that allow iOS to dynamically convert and render glTF assets on-the-fly.

The Role of Model I/O and RealityKit

When an iOS device encounters a .gltf or .glb file designated for Quick Look, the operating system routes the file through Apple’s low-level 3D graphics ingestion framework, Model I/O (MDLAsset), before passing it to RealityKit.

Model I/O acts as an intermediate parsing engine. It reads the structural JSON hierarchy of a glTF file, unpacking:

  • Node hierarchies and transformations
  • Vertex attributes (positions, normals, tangents, texture coordinates)
  • Index buffers defining geometric primitives

Rather than saving a static .usdz archive to physical storage, the system instantiates an in-memory scene representation. This intermediate representation translates Khronos Group specifications into Apple’s native scene graph components.

PBR Material Mapping to UsdPreviewSurface

The primary technical hurdle in real-time conversion is ensuring material fidelity. Both glTF 2.0 and USDZ utilize Physically Based Rendering (PBR) workflows centered on the Metallic-Roughness model, which simplifies cross-format translation.

During on-the-fly conversion, the system maps glTF PBR material inputs directly to the UsdPreviewSurface shader specification:

  • Base Color: Mapped to USD diffuseColor.
  • Metallic-Roughness: glTF packs roughness into the green channel and metallic into the blue channel of a single texture. Quick Look extracts and splits or routes these channels to USD's roughness and metallic inputs.
  • Normal Maps: Inverted if necessary to match the expected coordinate space (OpenGL vs. DirectX conventions) and piped into the USD normal input.
  • Occlusion and Emissive: Bound directly to corresponding ambient occlusion and emissive color nodes.

This deterministic mapping allows the conversion pipeline to bypass heavy shader recompilation, preserving GPU performance.

Geometry, Draco Decompression, and Rigging

glTF files often utilize Draco compression to reduce payload sizes for web transmission. To render these files on-the-fly:

  1. Buffer Decompression: Quick Look utilizes integrated Draco decoders to decompress geometric arrays directly into system RAM.
  2. Coordinate Normalization: glTF natively uses a right-handed, Y-up coordinate system, which aligns directly with USD and ARKit spaces, minimizing matrix reorientations.
  3. Skeletal Animation and Morph Targets: Joint hierarchies, skinning weights, and inverse bind matrices defined in glTF are mapped into USD’s UsdSkel schema. Keyframe animations driving translation, rotation, and scale channels are converted into uniform time-sampled USD animation curves.

Memory Optimization and Execution

Because Quick Look operates as an embedded extension within host applications like Safari, it runs under strict memory limits. The on-the-fly conversion minimizes memory overhead by:

  • Streaming binary buffers directly from disk or network cache without intermediate disk writes.
  • Dynamically generating mipmaps using the GPU rather than pre-allocating texture memory.
  • Instantiating the final scene directly as an entity within RealityKit, bypassing the need to generate an actual physical .usdz container.

Once the memory graph is established, Quick Look hands the scene over to ARKit for surface detection, lighting estimation, and real-time interaction.