Gary Marcus Urges Industry to Question the Engineering Behind AI Benchmarks
GaryMarcus · x · 2026-08-02
Reacting to recent impressive AI test results, Gary Marcus echoed an industry insider's deep skepticism: rather than just focusing on benchmark scores, there should be greater attention on the specific engineering processes achieving these breakthroughs. For instance, is it purely due to base model capability, or refined testing harnesses and post-training methods? The industry urgently needs more transparency regarding evaluation scaffolding and post-training details.
Related event: OpenAI Math Breakthrough Questioned by Gary Marcus and Others(38 posts)→
More from Models
- Gemini 4 Carbon Review: Stronger Coder Than Argon, but Fable 5.2 Raised the Bar — every · 2026-11-14
- Gemini 4 Krypton reportedly hits Opus 5.5-level performance — feblet · 2026-11-14
- Nathan Lambert: Jensen Huang's exit is a decapitation for Nvidia's open-model bet — natolambert · 2026-11-14
- Leak: Google 'krypton' model coming next week, Fable 5.2-level but cheaper — imjustnewatai · 2026-11-14
- OpenAI dissolves robotics team again as GPT-6 Astra can control robots directly — The Decoder · 2026-11-14
- Users report Sol 6.1 has become 'incredibly stupid' lately, making rookie mistakes — graham_dev · 2026-11-14