Convert Screen Coordinates to 3D in Three.js

Translating 2D screen coordinates from a mouse click into 3D world coordinates is a fundamental task in interactive 3D web development. This article provides a step-by-step guide on how to use the Three.js Raycaster to project a 2D mouse position into a 3D scene, detect which objects are clicked, and retrieve the exact 3D coordinates of the intersection point.

To convert 2D screen coordinates to 3D space, you must first capture the mouse click event, normalize the coordinates, cast a ray from the camera through those coordinates, and find where that ray intersects your 3D objects.

Step 1: Normalize the Mouse Coordinates

The browser measures mouse events in pixels, starting from the top-left corner (0,0) to the bottom-right. Three.js, however, expects coordinates in Normalized Device Coordinates (NDC) space, where both the X and Y axes range from -1 to 1, with (0,0) being the center of the screen.

Use the following formula to convert pixel coordinates to NDC:

const pointer = new THREE.Vector2();

window.addEventListener('click', (event) => {
    // Calculate pointer position in normalized device coordinates (-1 to +1)
    pointer.x = (event.clientX / window.innerWidth) * 2 - 1;
    pointer.y = -(event.clientY / window.innerHeight) * 2 + 1;
});

Step 2: Set Up the Raycaster

Once you have the normalized coordinates, you need to use the THREE.Raycaster to cast a ray from the camera through the mouse position on the screen.

const raycaster = new THREE.Raycaster();

Inside your click event listener, update the raycaster with the normalized coordinates and your active camera:

// Update the casting ray with the camera and pointer position
raycaster.setFromCamera(pointer, camera);

Step 3: Calculate Intersections and Retrieve the 3D Point

With the raycaster updated, you can search for intersections. You can check against the entire scene or a specific array of 3D objects.

// Calculate objects intersecting the picking ray
const intersects = raycaster.intersectObjects(scene.children, true);

if (intersects.length > 0) {
    // The first intersection is the closest one
    const firstIntersection = intersects[0];
    
    // Retrieve the 3D world coordinates of the intersection point
    const worldPosition = firstIntersection.point;

    console.log(`3D World Coordinates: X: ${worldPosition.x}, Y: ${worldPosition.y}, Z: ${worldPosition.z}`);
}

Complete Implementation Example

Here is how these steps fit together in a complete event listener:

const raycaster = new THREE.Raycaster();
const pointer = new THREE.Vector2();

window.addEventListener('click', (event) => {
    // 1. Normalize mouse coordinates
    pointer.x = (event.clientX / window.innerWidth) * 2 - 1;
    pointer.y = -(event.clientY / window.innerHeight) * 2 + 1;

    // 2. Set the raycaster from the camera
    raycaster.setFromCamera(pointer, camera);

    // 3. Find intersections
    const intersects = raycaster.intersectObjects(scene.children, true);

    if (intersects.length > 0) {
        const hitPoint = intersects[0].point;
        // 'hitPoint' is a THREE.Vector3 representing the 3D coordinate in the world
        console.log("Clicked at 3D point:", hitPoint);
    }
});

By using this approach, you can accurately determine where a user is clicking in your 3D environment, enabling interactions like object selection, placement, and movement.