Gemini 4 Hydrogen early tests: 5pp more accurate, half the time, 12% fewer tokens
demishassabis · x · 2026-11-13
Box CEO Aaron Levie shares early-access results for Gemini 4 Hydrogen on the Box AI Complex Work Eval: 5 percentage points better accuracy than Gemini 3.8 Flash, tasks done in less than half the time with 12% fewer tokens. Charts show 79% vs 74% on the full dataset, with the public sector subset jumping to 95%. Hydrogen rolls out across AI Studio, Gemini API, the Gemini app, Search, and Antigravity for paid tiers; Box AI Studio will let customers build custom agents with it. Demis Hassabis amplified the post.
More from Models
- OpenAI's $0.11 model beats the priciest LLMs in new Slack agent benchmark — heyshrutimishra · 2026-11-13
- Dean Ball: Open-ended long-horizon domains hit a wall near human level — verifiability is the crux — deanwball · 2026-11-13
- JevBench Splits Open-Weights Models and Hosted APIs onto Separate Leaderboards — airesearch12 · 2026-11-13
- JevBench update: top three within cost and speed caps are all open weights — airesearch12 · 2026-11-13
- Group chat take: Gemini 3.5 Flash-Lite best-value translator, Gemini 4 Carbon best Chinese writer — vista8 · 2026-11-13
- Every: Anthropic's Fable 5.2 builds production-grade software unattended overnight — every · 2026-11-13