How Linux Uses Looking Glass for Low Latency VMs
This article explains how the Linux operating system employs the open-source Looking Glass application to achieve ultra-low latency display streaming from a virtual machine (VM) to the host desktop. By bypassing traditional network-based capture stacks and external hardware requirements, Linux utilizes shared host memory, standard kernel virtualization features, and direct frame grabbing to render the guest's display directly in a native host window with near-zero performance loss.
The Foundation: GPU Passthrough and VFIO
In a standard virtualization setup using Linux's Kernel-based Virtual Machine (KVM) and QEMU, dedicated gaming or compute workloads often rely on VFIO (Virtual Function I/O) GPU passthrough. This assigns a physical graphics card entirely to the guest VM. Traditionally, viewing the output of this secondary GPU required plugging in a second physical monitor or using hardware KVM switches, as standard hypervisor virtual display adapters lack the 3D performance needed for high-end applications.
Inter-VM Shared Memory (IVSHMEM)
Looking Glass solves the display issue by using Inter-VM Shared
Memory (IVSHMEM) rather than network protocols like RDP or VNC. Linux
facilitates this by allocating a dedicated block of RAM using the
kvmfr (KVM FrameRelay) kernel module or standard POSIX
shared memory (/dev/shm).
Both the Linux host operating system and the guest virtual machine map this exact region of physical memory into their respective address spaces:
- The Linux host configures a QEMU parameter creating a shared memory device.
- The guest OS sees this memory as a dedicated PCI device.
Guest-Side Frame Capture
Inside the guest operating system (typically Windows), the Looking Glass host application runs in the background. It hooks directly into the graphics subsystem using native low-overhead capture APIs, such as Microsoft's Desktop Duplication API (DXGI).
As soon as the passthrough GPU renders a frame, the guest capture application immediately copies the raw framebuffer directly into the shared memory interface. Because this process writes uncompressed frames directly to local host RAM, it avoids the processing latency, frame drops, and visual compression artifacts caused by hardware video encoders like NVENC or H.264/HEVC.
Host-Side Frame Rendering
On the Linux desktop, the Looking Glass client application monitors
the shared memory block. Using the kvmfr kernel module, the
host client accesses the shared frame data directly via DMA (Direct
Memory Access):
- Zero-Copy Reading: The Linux client detects newly written frames in the shared memory pool.
- Hardware Acceleration: The client renders these raw frames into a standard desktop window utilizing modern graphics APIs like Vulkan or OpenGL on the host's primary GPU.
- Display Synchronization: Looking Glass employs frame pacing algorithms to match the host monitor's refresh rate, supporting variable refresh rate technologies (G-Sync and FreeSync).
Input Redirection via SPICE
Alongside video streaming, seamless interaction requires zero-latency input handling. Linux utilizes the SPICE (Simple Protocol for Independent Computing Environments) protocol solely for mouse and keyboard events. When the mouse pointer enters the Looking Glass window on the Linux host, the client captures input events and channels them directly through the local SPICE server instance into the VM, automatically synchronizing mouse movement, keyboard strokes, and clipboard contents without introducing input lag.