Fable 5.2's Puzzle Skills Look Suspiciously Good, Reciting Outdated MazeBench Solution

patience_cave · x · 2026-11-16

The MazeBench author observed something suspicious: on a difficult 3D level, Fable 5.2 produced the solution to an older version of MazeBench — the level was updated two weeks ago, making the old solution invalid, yet the model still answered with it.

Overall Fable 5.2 solves long puzzles in a single response, beats levels in minutes that took hours to design, and shows stronger 3D spatial reasoning than GPT-6 Astra — but this incident hints at possible training-data contamination.

Related event: Fable 5.2 Tops MazeBench Token Efficiency but May Have Memorized Answers(2 posts)→

Original post →

More from Models

Models channel →