Streaming Geometry with Memory-Mapped Files on NVMe

Memory-mapped files allow game engines to treat persistent storage directly as virtual memory, enabling lightning-fast streaming of static geometry from modern NVMe drives. By eliminating user-space buffer copies, leveraging kernel-level paging, and aligning data for direct-to-GPU uploads, developers can eliminate loading screens and stream high-density meshes smoothly. This article details the specific techniques that combine memory-mapping (mmap on POSIX or CreateFileMapping/MapViewOfFile on Windows) with NVMe capabilities to achieve instant static geometry streaming.

Zero-Copy Binary Layouts and Page Alignment

Traditional asset loading reads data into an intermediary CPU buffer, parses it, and copies it to a vertex buffer. Memory-mapped streaming eliminates this intermediary phase through structured binary compilation:

  • Pre-baked GPU-Ready Formats: Geometry is stored on disk in the exact binary format expected by the GPU (interleaved or non-interleaved vertex attributes, 16-bit/32-bit index buffers) with no runtime serialization, endian conversion, or pointer patching.
  • 4KB/2MB Boundary Alignment: Assets are aligned to 4 KB operating system page boundaries or 2 MB huge page boundaries. Because NVMe drives read in hardware block sizes (typically 4 KB sectors), aligning vertex and index data ensures that accessing an asset triggers optimal block reads without partial-page wastage.

Virtual Address Space Reservation with Sparse Commitments

64-bit systems provide a 128 TB virtual address space, allowing game engines to reserve vast contiguous regions of virtual memory without consuming physical RAM:

  • Pointer Stability: The engine maps an entire multi-gigabyte geometry package into a single contiguous virtual address space reservation (MEM_RESERVE on Windows or mmap with PROT_NONE on Linux).
  • Sparse Mapping: As chunks of geometry enter the visibility range, the engine maps specific subregions to the physical file view. Meshes can use absolute offsets and deterministic indexing relative to the base pointer, completely bypassing pointer swizzling or internal hash-map lookups.

Proactive Kernel Hinting and Asynchronous Prefetching

Relying on standard on-demand page faults can cause micro-stutters if the rendering thread touches unpaged memory. Engines prevent this using low-overhead OS paging hints:

  • POSIX madvise and Windows PrefetchVirtualMemory: When camera frustum checks or spatial acceleration structures (such as BVH nodes) detect approaching geometry, the engine issues MADV_WILLNEED or invokes PrefetchVirtualMemory.
  • NVMe Parallel Queue Saturation: Modern NVMe drives operate best under high queue depths across multiple hardware queues. Asynchronous prefetch hints allow the OS kernel to saturate these queues, pulling several mesh blocks into page cache simultaneously without blocking the main engine loop.

Amortizing Page Faults via Background Worker Probing

Because a hard page fault on unmapped memory suspends the calling thread until the NVMe controller delivers the data, synchronous touches on the main or render threads cause frame drops:

  • Touch-Paging Threads: High-priority worker threads "touch" mapped memory addresses (reading a single byte per 4 KB page) several frames before an asset enters the camera frustum.
  • Fault Absorption: The inevitable NVMe read latency and page fault handling are absorbed entirely by background worker threads. Once the read completes and the page table updates, the main thread can access the mesh with guaranteed zero-latency execution.

Direct GPU Uploads from Mapped System Pages

Once geometry data resides in the OS page cache via memory mapping, it must travel efficiently to GPU VRAM:

  • Direct DMA via Staging Buffers: Engines copy mapped pointers directly to write-combined CPU-visible GPU staging buffers (D3D12_HEAP_TYPE_UPLOAD or VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) using vectorized AVX-512 or SIMD copy instructions. This minimizes the pipeline to a single copy operation directly from cache to PCIe bus.
  • Bypassing CPU Page Cache via DirectStorage: When higher throughput is needed, techniques shift from standard kernel mmap to low-level APIs like Microsoft DirectStorage or Vulkan direct memory extensions. These utilize NVMe bypass paths, scheduling Direct Memory Access (DMA) transactions that pull data straight from the physical NVMe storage into GPU memory, eliminating CPU intervention and cache pollution entirely.