After MazeBench gets benchmaxxed, developers turn to secretive AI benchmarks
amplifiedamp · x · 2026-11-17
MazeBench's creator notes the benchmark is being benchmaxxed and argues benchmarks must be kept secret to reflect reality over any reasonable timespan. amplifiedamp adds their next AI detection eval won't be public. The thread captures a growing shift toward private evals as public benchmarks get gamed.
More from Models
- Tencent's Hunyuan Hy4 flash goes free on Nous Portal for two weeks — Teknium · 2026-11-17
- Anthropic bans user for asking a question, sparking 'AI is a utility, can't cut it off' backlash — mark_k · 2026-11-17
- Dev praises Gemini 4 carbon as his favorite model despite 20-minute waits — willcb · 2026-11-17
- Qwen 3.8 27B vs Claude Fable 5.2: a $0 local model (65 vs 20 on AA Index) and which jobs it can take over — alex_verem · 2026-11-17
- Blogger finds Astra skips steps on research tasks while carbon dutifully finishes them — lxfater · 2026-11-17
- Reuters exclusive: China taps US AI models via distillation to train homegrown rivals — ns123abc · 2026-11-17