RealityBench Benchmarks VLMs on Documenting Physical Work, Finds Weak Spatial Reasoning
relantic · hf · 2026-10-20
Researchers released RealityBench, a benchmark of simulated environments testing how well vision-language models can document human physical work.
- Findings: VLMs are decent at text-based documentation but struggle with measurement reading, spatial reasoning, and acceptability judgment.
- Cost-performance chart: top models (Gemini 4 Argon, Claude Fable 5.1, GPT-6 Astra) score roughly 80–85%+ at $1–$100 price points, while cheaper open models like Gemma 4 31B and LFM2.5-VL-3B reach 65–79% at far lower cost.
The benchmark is useful for gauging VLM readiness for real-world physical work documentation.
More from Models
- Claude users report subscription limits running out much faster than before — prasenx · 2026-10-20
- User reports ChatGPT todos silently vanish between messages, threatens switch to Grok — koltregaskes · 2026-10-20
- doubleAI, co-founded by Mobileye CEO, claims its model cuts compute costs by 67% — pstAsiatech · 2026-10-20
- Longtime Anthropic critic returns: Claude 5.5 lineup is the best in months — intellectronica · 2026-10-20
- Vals AI: DeepSeek V4.1 Flash 1017 beats GPT-6 Sol on offensive cybersecurity benchmark — Nunki08 · 2026-10-20
- ModelScope releases MemT-1.0: models with built-in persistent memory, 1.86x faster at 128K context — kalyan_kpl · 2026-10-20