How Mip-NeRF Eliminates Aliasing With Conical Frustums

This article provides a comprehensive overview of how Mip-NeRF, an advancement in 3D neural rendering artificial intelligence, solves the critical issue of aliasing. We will explore the limitations of traditional single-ray casting in original Neural Radiance Fields (NeRF), explain the mechanics of casting 3D conical frustums, and detail how Integrated Positional Encoding (IPE) enables continuous, multi-scale representations that keep 3D renders sharp and artifact-free at any viewing distance.

The Problem with Traditional NeRF Ray Casting

To understand Mip-NeRF, you must first understand the flaw in the original NeRF architecture. Traditional NeRF renders 3D scenes by marching a single, infinitely narrow ray per pixel from the camera into the 3D space. It samples discrete points along this ray and feeds their coordinates into a neural network to predict color and volume density.

This approach fails when the camera moves closer to or further away from the subject—a problem known as aliasing. Because a single ray has no volume, it only samples an infinitesimally small point in space. When rendering a low-resolution image or viewing an object from far away, a single pixel on a screen actually covers a large area of the 3D scene. Sampling just one point within that large area completely misses the surrounding context, resulting in jagged edges, pixelation, and shimmering artifacts when the camera moves.

Casting 3D Conical Frustums

Mip-NeRF resolves this issue by changing the fundamental geometry of how the AI looks at a scene. Instead of shooting a one-dimensional ray through the center of each pixel, Mip-NeRF casts a three-dimensional cone.

The apex of this cone starts at the camera's center of projection and widens as it travels outward through the pixel and into the 3D scene. The radius of the cone is directly determined by the physical size of the pixel on the image plane. Instead of sampling flat points along a line, Mip-NeRF slices this cone into segments along its length. These individual 3D volumetric slices are called conical frustums—essentially cones with their pointy tips chopped off.

By utilizing conical frustums, the AI no longer asks, "What is at this exact point?" Instead, it asks, "What is the average appearance of everything inside this specific 3D volume?"

Approximating Frustums with Multivariate Gaussians

Passing a complex geometric shape like a conical frustum directly into a neural network is computationally expensive and mathematically difficult. Mip-NeRF bypasses this bottleneck by approximating the shape of each conical frustum using a multivariate Gaussian distribution.

A 3D Gaussian represents a soft, probabilistic cloud in space, defined by its center point (mean) and its spread in three dimensions (covariance). Mip-NeRF calculates the exact mean and covariance that closely match the volume of the conical frustum. This mathematical shortcut translates a hard geometric volume into a compact set of numbers that a neural network can easily process.

Integrated Positional Encoding (IPE)

In a standard NeRF, 3D points are translated into higher-dimensional frequencies using Positional Encoding (PE), which helps the neural network learn fine, high-frequency details like sharp edges and textures.

Because Mip-NeRF deals with volumes (Gaussians) rather than single points, it introduces Integrated Positional Encoding (IPE). Instead of encoding a single coordinate, IPE calculates the expected value (the integral) of the positional encoding across the entire Gaussian volume.

This integration acts as an automatic, continuous anti-aliasing filter:

Close-up Views (Narrow Cones): When a camera is close to an object, the conical frustum is very small. IPE retains the high-frequency encodings, allowing the neural network to render sharp, intricate details.
Distant Views (Wide Cones): When the camera pulls back, the conical frustum becomes very wide, encompassing a large part of the scene. Over this large volume, high-frequency waves cancel each other out during integration, leaving only low-frequency information. This naturally blurs the input features, forcing the network to output a smooth, pre-filtered representation of that space.

The Result: Multi-Scale Representation

By replacing rays with conical frustums and point encoding with volume encoding, Mip-NeRF inherently understands scale. The neural network learns to associate narrow frustums with fine details and wide frustums with smoothed, averaged appearances. This mimics the classic computer graphics technique of "mipmapping," where textures are pre-filtered at multiple resolutions, but applies it continuously in 3D space. Consequently, Mip-NeRF completely eliminates aliasing artifacts while training a single, unified neural network that can render scenes accurately at any resolution or camera distance.