Native video inference beats sampled frames, but OpenRouter providers don't support it yet
spillai · x · 2026-09-08
Responding to the egocentric VLM eval thread, spillai (vlmrun) notes that evaluating via sampled images instead of native video likely hurts performance, and that OpenRouter providers generally don't support native video out of the box — he saw real discrepancies while running evals. His vlmrun gateway will host more VLMs/world models for robotics evals, exposing fps, dynamic sampling, and resolution controls relevant to these use cases.
More from Models
- Repeating instructions in prompts still makes models adhere better, like it's 2023 — tokenbender · 2026-09-08
- Model slips on a science knowledge test, author calls it acceptable — felpix_ · 2026-09-08
- Users game Tibo's token reset: spend 80% fast, stagger reset cycles to dodge surprise resets — tinyfool · 2026-09-08
- Economists in the top 10% of AI use see no step change from GPT 5.6 to 6 — aniketapanjwani · 2026-09-08
- Mathematician: Astra solved two of my unpublished theorems in ~60 hours each, $20k in API — basedjensen · 2026-09-08
- Drop-in Claude.md patch claims to fix Claude's degraded prose in Opus and Fable — myriable · 2026-09-08