Vector Institute's 3DSPA Measures and Improves Physical Realism in Video Models

Kelsey Allen, a researcher at the Vector Institute, presented at the 2026 AI Summit how cognitive science can help fill AI's gaps: her proposed 3DSPA (3D Semantic Point Autoencoder) can measure the physical realism of generated video and serve as a reinforcement learning reward signal, delivering measurable improvements in video models' physical behavior.

Confirmed

Background problem: modern large models excel at knowledge representation but lack physical reasoning; Virtual Tools game experiments showed that physical tasks humans solve via mental simulation are handled poorly by current state-of-the-art AI systems
Method: 3DSPA represents videos via a 3D semantic point autoencoder, which is used to measure the physical realism of video
Results: when used as a reinforcement learning reward signal, 3DSPA made gravity behave correctly in generated videos, with objects no longer passing through solid obstacles
Venue: Vector Institute 2026 AI Summit and Career Fair; Kelsey Allen is a faculty member at the institute

Why it matters

Cognitive science offers a new path for evaluating and improving AI: using human mental simulation as a reference exposes AI's physical reasoning weaknesses and points toward fixes
It provides generative video models with a measurable physical realism metric and an optimizable reward signal, rather than relying on subjective human judgment

2026-10-08 ~ 2026-10-08 · 5 related posts

Primary sources

Vector Institute introduces 3DSPA: a 3D semantic point autoencoder to measure physical realism in generated video — VectorInst · 2026-10-08 22:57 (UTC+08:00)
3DSPA as an RL reward signal measurably improves physical realism: gravity works, objects stop passing through barriers — VectorInst · 2026-10-08 22:57 (UTC+08:00)

Vector Summit: how cognitive science helps us build better AI — VectorInst · 2026-10-08
Cognitive scientist Kelsey Allen: human mental simulation exposes the physical reasoning limits of current AI — VectorInst · 2026-10-08
[source] Vector Institute introduces 3DSPA: a 3D semantic point autoencoder to measure physical realism in generated video — VectorInst · 2026-10-08
[source] 3DSPA as an RL reward signal measurably improves physical realism: gravity works, objects stop passing through barriers — VectorInst · 2026-10-08
Vector's 3DSPA reward signal fixes physics violations in generative video models — VectorInst · 2026-10-08