CogVideoX Physics Consistency Detection Experiment
Divine-Demon7 · reddit · 2026-07-20
The author is working on an indie project called Haga, aimed at building a physics consistency detector for generated videos and robotics policy simulations. Here, they report a pre-registered, held-out set experiment validating the statichover failure mode.
Experimental Setup
- Model: THUDM/CogVideoX-5b-I2V
- Data: 3 perspectives × seeds 2, 3, and 4, totaling 9 videos
- Thresholds and inclusion rules frozen prior to generation
- Pipeline: RGB → CoTracker3 → position-only VIDEOCHECKS
- Discovery seeds 0–1 separated from held-out seeds 2–4
Results
- Held-out set hit rate: 9/9, 100%
- Wilson 95% CI: [0.701, 1.000]
- All 9 videos triggered statichover
- Real Physics-IQ videos did not trigger alerts under the same configuration
Stated Limitations
- Only one I2V model was tested
- Only covers a single "balls and blocks falling" scenario
- Only covers one known failure mode
- Results cannot be generalized to the overall quality of CogVideoX
The author is actively seeking critique on what physical violations position tracking might miss, whether statichover is defined too narrowly, and what public video artifacts should be tested next.
More from Multimodal
- Lumara AI Film Festival Comes to NYC Oct 26, Top AI Filmmakers to Compete — 0xAllen_ · 2026-09-11
- Pterodactyl Detective: An AI-Generated Proof-of-Concept Trailer — PterodactylDetective · 2026-09-11
- Imperium Game Trailer Showcases AI Video Generation — keaslenyt · 2026-09-11
- FLUX.2 Klein Drifts Hard on Character Expressions While Free Gemini Holds Likeness — wacomlover · 2026-09-11
- Tencent Hunyuan releases AuK code and weights on GitHub with ComfyUI and fine-tuning support — aigclink · 2026-09-11
- Creator turns Bahamut vs Tiamat rivalry into an AI cinematic battle with Midjourney, GPT Image 2 and Seedance — azed_ai · 2026-09-11