CogVideoX Physics Consistency Detection Experiment
Divine-Demon7 · reddit · 2026-07-20
The author is working on an indie project called Haga, aimed at building a physics consistency detector for generated videos and robotics policy simulations. Here, they report a pre-registered, held-out set experiment validating the statichover failure mode.
Experimental Setup
- Model: THUDM/CogVideoX-5b-I2V
- Data: 3 perspectives × seeds 2, 3, and 4, totaling 9 videos
- Thresholds and inclusion rules frozen prior to generation
- Pipeline: RGB → CoTracker3 → position-only VIDEOCHECKS
- Discovery seeds 0–1 separated from held-out seeds 2–4
Results
- Held-out set hit rate: 9/9, 100%
- Wilson 95% CI: [0.701, 1.000]
- All 9 videos triggered statichover
- Real Physics-IQ videos did not trigger alerts under the same configuration
Stated Limitations
- Only one I2V model was tested
- Only covers a single "balls and blocks falling" scenario
- Only covers one known failure mode
- Results cannot be generalized to the overall quality of CogVideoX
The author is actively seeking critique on what physical violations position tracking might miss, whether statichover is defined too narrowly, and what public video artifacts should be tested next.
More from Multimodal
- Open-source local studio bundles image, video, music and AI movie maker offline — lelus78 · 2026-10-26
- Techhalla shares full 2D-to-3D workflow with Opus 5.5 and Kling 4.0 prompts — techhalla · 2026-10-26
- Rebuilding real events as photoreal SF digital twins, filmed cinematically from a phone — bilawalsidhu · 2026-10-26
- How to turn Claude Code into a video editor: a full reel in 19 minutes — PrajwalTomar_ · 2026-10-26
- "Recursive eyes" prompt idea produces mesmerizing spiral of infinitely recurring faces — LudovicCreator · 2026-10-26
- Black Forest Labs open-sources 5.6B FLUX 3 Image Klein, +91 Elo with 38% fewer params — dl_weekly · 2026-10-26