Fable 5.2's Puzzle Skills Look Suspiciously Good, Reciting Outdated MazeBench Solution
patience_cave · x · 2026-11-16
The MazeBench author observed something suspicious: on a difficult 3D level, Fable 5.2 produced the solution to an older version of MazeBench — the level was updated two weeks ago, making the old solution invalid, yet the model still answered with it.
Overall Fable 5.2 solves long puzzles in a single response, beats levels in minutes that took hours to design, and shows stronger 3D spatial reasoning than GPT-6 Astra — but this incident hints at possible training-data contamination.
Related event: Fable 5.2 Tops MazeBench Token Efficiency but May Have Memorized Answers(2 posts)→
More from Models
- Fable 5.2 Leads MazeBench Token Efficiency; Billion-Token Runs Underway — patience_cave · 2026-11-16
- Fable 5.2 Tops MazeBench With 21% After 90 Hours and $6,000 in Credits — patience_cave · 2026-11-16
- Tencent Hy4 flash and InternLM Sage-S2.1-Flash both released, tool-calling tests requested — MajesticPuzzler · 2026-11-16
- User Shares Coral Reef Creation Labeled 'Qwen 4 Max (xhigh)', an Unannounced Model Name — intellectronica · 2026-11-16
- Stealth Model 'silicon-2' Surfaces in Devin, Suspected to Be Anthropic's Next Model — kimmonismus · 2026-11-16
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16