Custom Transparent Sorting in Three.js

This article explains how to implement custom sorting for transparent and opaque objects in Three.js using the setTransparentSort and setOpaqueSort methods on the WebGLRenderer. You will learn how the default sorting mechanism works, how to override it with your own logic, and how to structure custom sorting functions to resolve depth-fighting and transparency rendering issues.

Understanding Three.js Default Sorting

By default, Three.js handles object sorting automatically to optimize rendering performance and visual accuracy: * Opaque objects are sorted from front to back (closest to the camera first). This maximizes depth-buffer occlusion, preventing the GPU from rendering pixels that will be covered by closer objects. * Transparent objects are sorted from back to front (furthest from the camera first). This is necessary for alpha blending to render correctly, as background transparent objects must be drawn before foreground transparent objects.

However, default sorting calculates distance based on the center of the object’s bounding sphere. For large, complex, or overlapping geometries, this calculation can fail, resulting in rendering artifacts.

The WebGLRenderer Sorting Methods

To override the default sorting algorithms, the WebGLRenderer provides two methods:

These methods accept a custom comparison function. If you pass null to these methods, the renderer reverts to its default internal sorting algorithms.

Anatomy of the Render Item

The custom sorting function behaves similarly to a standard JavaScript Array.prototype.sort() comparison function. It takes two arguments, a and b, which represent “render items.”

Each render item is an object containing metadata about the element being rendered. The key properties available on these items are:

Implementing a Custom Transparent Sort

To set up a custom sorting function, define your comparison logic and pass it to the renderer.

Here is how to set up a custom transparent sort that prioritizes a custom property in the object’s userData before falling back to the default depth-based sorting:

// 1. Initialize your WebGLRenderer
const renderer = new THREE.WebGLRenderer();

// 2. Define and apply the custom sorting function
renderer.setTransparentSort(function(a, b) {
    // Check for a custom sort priority defined in the object's userData
    const priorityA = a.object.userData.sortPriority || 0;
    const priorityB = b.object.userData.sortPriority || 0;

    if (priorityA !== priorityB) {
        // Render objects with higher priority on top (drawn later)
        return priorityA - priorityB; 
    }

    // Fallback: Default back-to-front rendering (based on camera-space Z depth)
    // In Three.js, higher z values in the render list mean closer to the camera.
    // To render back-to-front, we want larger z values (closer) to be drawn last.
    return b.z - a.z;
});

To use this custom sorting in your scene, assign the sortPriority property to your meshes:

const glassSphere = new THREE.Mesh(geometry, transparentMaterial);
glassSphere.userData.sortPriority = 2; // Will render after lower priority objects

const waterPlane = new THREE.Mesh(planeGeometry, waterMaterial);
waterPlane.userData.sortPriority = 1; 

scene.add(glassSphere);
scene.add(waterPlane);

Creating a Stable Custom Sort

If your custom sorting criteria return 0 (indicating equal priority), the relative order of the objects may become unstable during rendering, causing flickering. To ensure a stable sort, always provide a definitive fallback, such as the unique object ID:

renderer.setTransparentSort(function(a, b) {
    // Custom logic
    const diff = a.object.userData.customDepth - b.object.userData.customDepth;
    
    if (diff !== 0) {
        return diff;
    }
    
    // Stable fallback using unique object IDs
    return a.id - b.id;
});