Gemini V4-Pro Disappoints in Tests, Suspected to Be Hampered by Internal Distillation
teortaxesTex · x · 2026-08-13
A developer expressed disappointment with Gemini V4-Pro's performance, noting it functions essentially like V4-Flash-0731 with pass@2 (best of two attempts).
They suspect that Google's internal distillation techniques within the same model family equalize the scales, which consistently makes the lighter Flash version appear more impressive in practice. They also predict the model will score around 70 on the ARC-2 benchmark.
More from Models
- Devs say LLM coding mistakes are a different class: overcomplication, inferred limits, worse at higher reasoning — Mu5_ · 2026-10-16
- ChatGPT UI screenshot reveals Wallet coming soon plus Email, Calendar, Health and Finances integrations — jxnlco · 2026-10-16
- Why the next OpenAI and Anthropic models may still disappoint: no native audio or video input — imjustnewatai · 2026-10-16
- Thousands Hit by ChatGPT Standard Voice Mode Regression: Wrong Voice, Interruptions — shaunralston · 2026-10-16
- Gemini Argon 4 Quota Exhausted, Reset Countdown Shows Nearly 7 Days — ssh4net · 2026-10-16
- AINews: Meta ships 816B Muse Spark Open; Goodfire argues neural nets are already clean — Latent Space · 2026-10-16