Fable 5.2 reproduces outdated MazeBench solutions, sparking contamination suspicion
patience_cave · x · 2026-11-16
patiencecave reports Fable 5.2 solves puzzles suspiciously well: on a hard 3D level it produced the solution to an old version of MazeBench, updated weeks ago so the old fix no longer works—and the model stumbled there, though it could sometimes infer recent changes. xeophon and others ask: did Anthropic train on MazeBench? A classic suspected benchmark-contamination case.
More from Models
- Honesty test: GPT-6 Astra vs Claude Opus 5.5 tempted to fake a lab result — qinzytech · 2026-11-16
- 22 LLMs quizzed on Nvidia's new CEO: Meta wrong 68% of the time — gaganghotra_ · 2026-11-16
- Google AI Pro subscriber: available models seem random across Antigravity and Gemini web — crono760 · 2026-11-16
- User's $300 subscription generated $7,492 of usage — 25x value on Codex and Claude — theteknosaur · 2026-11-16
- Anthropic Cuts Vulnerability Disclosure Window to 30 Days for Critical Bugs — WeldPond · 2026-11-16
- Liquid AI's d1.2-ultra tops Artificial Analysis Decision Index at 75, d1.2 lands #6 — ArtificialAnlys · 2026-11-16