Filter Raycast Hits with Collision Groups in Ammo.js

In ammo.js, performing efficient raycasting often requires ignoring specific physics bodies, such as triggers, invisible boundaries, or the shooter’s own character. This article explains how to strategically filter raycast hits by leveraging Bullet’s native collision filtering system, using 32-bit integer bitmasks to define custom collision groups and masks for precise raycast queries.

Understanding Bitwise Collision Filtering

Ammo.js (a port of the Bullet physics engine) uses a bitwise system for collision filtering. Every rigid body is assigned two properties when added to the dynamics world: * Collision Group: A bitmask representing “what I am.” * Collision Mask: A bitmask representing “what I collide with.”

A collision (or raycast hit) is only registered if both of the following conditions are met: 1. The ray’s mask shares a common bit with the rigid body’s group: (rayMask & bodyGroup) != 0 2. The body’s mask shares a common bit with the ray’s group: (bodyMask & rayGroup) != 0

By default, Ammo.js assigns default groups and masks that make everything collide with everything. To filter your raycasts, you must explicitly define these bitmasks.

Step 1: Define Your Collision Groups

Start by defining your collision groups using 32-bit integer flags. Using bitwise shift operators makes this clean and easy to read.

// Define custom collision groups
const GROUP_DEFAULT   = 1 << 0; // 1
const GROUP_PLAYER    = 1 << 1; // 2
const GROUP_WALL      = 1 << 2; // 4
const GROUP_TRIGGER   = 1 << 3; // 8
const GROUP_DEBRIS    = 1 << 4; // 16

Step 2: Assign Groups and Masks to Rigid Bodies

When adding rigid bodies to your physics world, you must pass their respective group and mask as the second and third arguments to addRigidBody.

// A player body that collides with walls and triggers, but not debris
const playerGroup = GROUP_PLAYER;
const playerMask  = GROUP_WALL | GROUP_TRIGGER;
dynamicsWorld.addRigidBody(playerBody, playerGroup, playerMask);

// A wall body that collides with players and default objects
const wallGroup = GROUP_WALL;
const wallMask  = GROUP_PLAYER | GROUP_DEFAULT;
dynamicsWorld.addRigidBody(wallBody, wallGroup, wallMask);

Step 3: Configure the Raycast Callback

When performing a raycast using ClosestRayResultCallback or AllHitsRayResultCallback, you can set the m_collisionFilterGroup and m_collisionFilterMask properties on the callback object before executing the test.

Because Ammo.js uses WebIDL bindings, you access these properties using set_m_collisionFilterGroup() and set_m_collisionFilterMask().

// 1. Create ray vectors
const rayFrom = new Ammo.btVector3(0, 10, 0);
const rayTo = new Ammo.btVector3(0, -10, 0);

// 2. Create the raycast callback
const rayCallback = new Ammo.ClosestRayResultCallback(rayFrom, rayTo);

// 3. Define the ray's identity (What it is)
// Treat the ray as if it is coming from the player group
rayCallback.set_m_collisionFilterGroup(GROUP_PLAYER);

// 4. Define the ray's filter (What it is allowed to hit)
// We want this ray to hit walls, but ignore triggers and debris
const rayMask = GROUP_WALL;
rayCallback.set_m_collisionFilterMask(rayMask);

Step 4: Perform the Ray Test

With the callback configured, execute the rayTest on your dynamics world. Only the bodies matching your mask constraints will be evaluated for a hit.

// Perform the raycast
dynamicsWorld.rayTest(rayFrom, rayTo, rayCallback);

if (rayCallback.hasHit()) {
    const hitBody = Ammo.castTo(rayCallback.get_m_collisionObject(), Ammo.btRigidBody);
    const hitPoint = rayCallback.get_m_hitPointWorld();
    
    console.log("Ray hit a valid physics body at:", hitPoint.x(), hitPoint.y(), hitPoint.z());
} else {
    console.log("Ray missed or was filtered out.");
}

// Clean up memory
Ammo.destroy(rayFrom);
Ammo.destroy(rayTo);
Ammo.destroy(rayCallback);

Summary of Best Practices