CogVideoX Physics Consistency Detection Experiment
Divine-Demon7 · reddit · 2026-07-20
The author is working on an indie project called **Haga**, aimed at building a physics consistency detector for generated videos and robotics policy simulations. Here, they report a pre-registered, held-out set experiment validating the `static_hover` failure mode. ### Experimental Setup - Model: `THUDM/CogVideoX-5b-I2V` - Data: 3 perspectives × seeds 2, 3, and 4, totaling 9 videos - Thresholds and inclusion rules frozen prior to generation - Pipeline: RGB → CoTracker3 → position-only `VIDEO_CHECKS` - Discovery seeds 0–1 separated from held-out seeds 2–4 ### Results - Held-out set hit rate: **9/9, 100%** - Wilson 95% CI: **[0.701, 1.000]** - All 9 videos triggered `static_hover` - Real Physics-IQ videos did not trigger alerts under the same configuration ### Stated Limitations - Only one I2V model was tested - Only covers a single "balls and blocks falling" scenario - Only covers one known failure mode - Results cannot be generalized to the overall quality of CogVideoX The author is actively seeking critique on what physical violations position tracking might miss, whether `static_hover` is defined too narrowly, and what public video artifacts should be tested next.
More from Multimodal
- Creator says he directed a full short film with Adobe Firefly AI Assistant — LudovicCreator · 2026-07-21
- Reddit users say Krea 2 Turbo regains strong facial expressions with bypass LoRAs — YentaMagenta · 2026-07-21
- AI music demo blends Suno v5.5, Reason Studios and Grok Imagine 1.5 — Kyrannio · 2026-07-21
- AI anime workflow article breaks storytelling into repeatable prompt steps — Aiden_Tech_Ai · 2026-07-21
- Claude is being pitched as a free workflow for viral YouTube Shorts scripts — Aiden_Tech_Ai · 2026-07-21
- A Bittensor game demo claims a two-person team built a playable 3D world in 30 days — markjeffrey · 2026-07-21