Sync Babylon.js Meshes with Matter.js 2D Physics

Synchronizing Babylon.js planar meshes with Matter.js rigid bodies allows developers to combine the visual fidelity of a 3D rendering engine with the predictable behavior of a lightweight 2D physics engine. To achieve this, you must set up an orthographic or plane-facing camera in Babylon.js, align the disparate coordinate systems of both engines, and continuously update the mesh transforms inside Babylon's render loop based on the rigid body data computed by Matter.js.

1. Reconcile the Coordinate Systems

Matter.js and Babylon.js use fundamentally different coordinate systems:

2. Configure the Babylon.js Camera

To match a 2D physics view accurately, configure a Babylon.js FreeCamera or UniversalCamera to face the XY plane from a distance, or use an orthographic projection:

const camera = new BABYLON.FreeCamera("camera2D", new BABYLON.Vector3(0, 0, -10), scene);
camera.setTarget(BABYLON.Vector3.Zero());

// Enable orthographic projection to eliminate perspective distortion
const ratio = canvas.width / canvas.height;
const orthoSize = 10; // Viewport height in Babylon units
camera.mode = BABYLON.Camera.ORTHOGRAPHIC_CAMERA;
camera.orthoTop = orthoSize / 2;
camera.orthoBottom = -orthoSize / 2;
camera.orthoLeft = (-orthoSize * ratio) / 2;
camera.orthoRight = (orthoSize * ratio) / 2;

3. Establish an Entity Mapping

Store a mapping between each Matter.js Body and its corresponding Babylon.js Mesh. A simple JavaScript object or class pair works effectively:

const physicsBindings = [];

function createBox(x, y, width, height) {
    // Create Matter.js 2D body
    const body = Matter.Bodies.rectangle(x, y, width, height);
    Matter.Composite.add(engine.world, body);

    // Create Babylon.js planar mesh (Plane or thin Box)
    const mesh = BABYLON.MeshBuilder.CreatePlane("boxMesh", {
        width: width / PIXELS_PER_METER,
        height: height / PIXELS_PER_METER
    }, scene);

    physicsBindings.push({ body, mesh });
}

4. Update Transforms in the Render Loop

In the Babylon.js render loop, step the Matter.js engine and synchronize each mesh's position and rotation with its associated physics body. Translate screen-space pixels into Babylon.js scene units and invert the Y and rotation axes:

const PIXELS_PER_METER = 50;
const screenWidth = canvas.width;
const screenHeight = canvas.height;

scene.onBeforeRenderObservable.add(() => {
    // Step the 2D physics engine
    Matter.Engine.update(engine, engine.timing.delta);

    // Sync meshes to bodies
    for (let i = 0; i < physicsBindings.length; i++) {
        const { body, mesh } = physicsBindings[i];

        // Convert Matter.js (X, Y) to Babylon.js (X, Y)
        const babylonX = (body.position.x - screenWidth / 2) / PIXELS_PER_METER;
        const babylonY = -(body.position.y - screenHeight / 2) / PIXELS_PER_METER;

        mesh.position.x = babylonX;
        mesh.position.y = babylonY;

        // Invert rotation angle for counter-clockwise orientation
        mesh.rotation.z = -body.angle;
    }
});

5. Managing Resizing and Disposals