Benchmarking AI Physics with Infant Psychology Paradigms

This article explores how developmental psychologists and AI researchers adapt infant violation-of-expectation (VoE) paradigms to evaluate the intuitive physics of video artificial intelligence models. By translating classic cognitive tests—specifically those measuring object permanence, gravity, and solidity—into digital benchmarks, scientists can directly compare machine perception with human infant cognition. The following sections break down the mechanics of the VoE paradigm, the digital adaptation of core physical concepts, and how AI "surprise" is mathematically quantified to measure a model's grasp of the physical world.

The Violation-of-Expectation (VoE) Paradigm

In developmental psychology, the Violation-of-Expectation paradigm relies on a simple premise: infants look longer at events that defy their intuitive understanding of the world. Because babies cannot verbally explain what they know, researchers measure their gaze duration when watching physically plausible events versus physically impossible "magic tricks." A significantly longer gaze at the impossible event indicates that the infant expected a different outcome, proving they possess an innate or early-learned model of how the physical world operates.

Translating Infant gazes into AI "Surprise"

Video AI models do not have eyes to gaze, so researchers must map the concept of human surprise onto mathematical metrics. Video models generally work by predicting future frames or latent representations based on past frames.

To measure VoE in AI, researchers feed the model paired video datasets: one showing a physically plausible scene and another showing a nearly identical scene with a single physical violation. The AI's "gaze duration" is replaced by its prediction error (or loss function). If an AI model understands intuitive physics, its prediction error will spike significantly when watching the impossible video compared to the plausible one. This spike represents computational surprise.

Benchmarking Object Permanence

Object permanence is the understanding that objects continue to exist even when they cannot be seen. In infant studies, a researcher might roll a ball behind a screen, remove the screen, and reveal that the ball has vanished.

To benchmark AI, researchers use 3D-rendered environments (like the IntPhys or InfLevel datasets) to recreate this exact scenario.

• Plausible Video: A red sphere rolls behind a brick wall and emerges on the other side.
• Impossible Video: A red sphere rolls behind a brick wall and never emerges, or the wall drops to reveal nothing behind it.

A physically intuitive AI model tracks the occluded object in its memory. When the object vanishes in the impossible video, the model registers a high prediction error, passing the benchmark. Models that rely purely on pixel-level patterns without understanding 3D space fail to show surprise, as they simply treat the disappearance as a standard color change on the screen.

Benchmarking Solidity and Continuity

Solidity refers to the principle that two solid objects cannot occupy the same space at the same time. In infant tests, a solid object is dropped onto a surface; in the trick condition, it passes right through the solid surface.

In AI benchmarking, this is tested using rigid-body simulations:

• Plausible Video: A tumbling block hits a solid wall and bounces back.
• Impossible Video: A tumbling block passes directly through the solid wall without altering the wall's structure.

For an AI to pass this benchmark, it must understand object boundaries and material density. If the AI model generates a high surprise score during the pass-through video, it demonstrates an internal representation of solidity.

Benchmarking Gravity and Support

Infants quickly learn that unsupported objects fall and that objects require a certain center of mass to remain stable on a ledge.

Researchers test AI on gravity and support by rendering scenarios involving ramps, tables, and falling objects:

• Plausible Video: A heavy block is pushed entirely off the edge of a table and falls to the floor.
• Impossible Video: A heavy block is pushed entirely off the edge of a table and hovers in mid-air, or accelerates upward.

By isolating gravity in these video pairs, researchers determine if the AI has learned a generalized rule about downward acceleration, rather than just memorizing the trajectories of specific objects in its training data.

Controlling for Visual Confounds

A major challenge in adapting these paradigms is ensuring the AI is reacting to the physical violation, not just sudden changes in pixel data. If an impossible video features a sudden flash or an erratic movement not present in the plausible video, the AI's prediction error will spike for the wrong reasons.

To solve this, psychologists and computer scientists carefully counterbalance the datasets. They ensure that the plausible and impossible videos share the exact same lighting, object speeds, and pixel-change rates up until the exact frame of the physical violation. Only when an AI consistently flags the violation across thousands of visually diverse, tightly controlled pairs can researchers conclude the model has achieved an infant-level grasp of intuitive physics.