LabyrinthBench: Measuring Agent Context Recall Shows Wiping History Wins
jwdeaver · reddit · 2026-08-07
LabyrinthBench is a new local-focused, judge-free benchmark designed to measure LLM context recall and currency under interference for multi-step agentic tasks.
- No LLM Judge Required: Uses deterministic mazes and logic gates (e.g., variable tracking, chained calculations) to objectively score models without subjective evaluation overhead.
- Targeted Pain Point: Simulates the common issue where models lose track of crucial information after hitting context limits or triggering compaction during long agent runs.
- Counter-intuitive Findings: In the first experiment on a 20-gate corridor with 13 local models, wiping the full chat history every turn and re-injecting only the model's recorded gate answers performed better in 7 out of 9 models. The same context-management trick can significantly improve some models while breaking others.
More from coding & agent
- Opus 5.5 + Three.js demo stuns with water physics, fish and blossoms — nikola_mr64990 · 2026-10-16
- Grok-built linear regression bot claims $508K PnL in 125 days on Polymarket — Roger_M_Taylor · 2026-10-16
- Karpathy's knowledge engine workflow: Claude Code turns raw notes into a self-updating wiki — nikola_mr64990 · 2026-10-16
- Sierra and Meta unveil Personal Agent Protocol with Zapier as launch partner — sujingshen · 2026-10-16
- ArcKit adds 102 AI-narrated short films for commands across 14 community overlays — mcraddock · 2026-10-16
- Personal agents live or die on chaining tasks, not single-task ability — sujingshen · 2026-10-16