Open RoboJev-27B judges robot success at 95.2% accuracy, 48x cheaper than GPT-6
Josef Liyanjun Chen · hf · 2026-10-27
RoboJev-27B, an open-weight vision-language judge for robot manipulation success, is now on Hugging Face. Using LoRA fine-tuning, calibrated token readouts and source-specific regularization, it hits 95.2% balanced accuracy on 268 held-out SO-101 runs, 0.890 AUROC on unseen labs, at $0.19 per 1,000 answers on one H100 (3.9 answers/s) — 48x cheaper than GPT-6 Astra and 10x cheaper than GPT-6.1 Sol while beating frontier models on accuracy. The author also projects that by 2035, 19B hours/year of robot video would need 9.5M full-time human labelers vs 21,164 GPUs with RoboJev-27B.
More from Embodied
- Toyota Pours $6.4B a Year Into Humanoid Robots, Targets 400,000 by 2028 — emmanuelvivier · 2026-10-27
- GPT-6 moves a camera to insert a flower, yet burns an egg: reflections on the Astra report — liuziwei7 · 2026-10-27
- Harness Device ditches paper box for premium UV-printed metal tin packaging — dee_hw · 2026-10-27
- $299 Desktop AI Assistant Intern 2 Adds iMessage, Ships Preloaded With OpenClaw Agent — dee_hw · 2026-10-27
- UBTECH opens humanoid robot factory producing one robot every 10 minutes — import_jmr · 2026-10-27
- Nebius × Positronic Physical AI Competition opens: top 10 sim policies run on real robot arm — lukas_m_ziegler · 2026-10-27