Recursive language models treat context as a REPL variable, lifting long-CoT accuracy from 2.6% to 45.4%

AI Engineer · youtube · 2026-09-09

Kevin Madura, director of advanced technology at AlixPartners, explains recursive language models (RLMs) in an AI Engineer talk.

Core idea: An RLM treats its context as an object in a Python REPL rather than tokens to attend over, so it can slice, compute, and iterate on input — e.g. summing twelve numbers scattered across 30k tokens via a few regex lines instead of direct attention.
Recursive delegation: It can delegate subproblems to another model (including itself with its own parameters), returning only the results that matter and avoiding context rot.
Contrast with today's stack: RAG stuffs the window until quality rots; agents/tool calls shuttle JSON with loosely coupled logic and execution — RLMs keep all three in one environment.
Results: Accuracy on a long chain-of-thought benchmark jumped from 2.6% to 45.4%, with the biggest gains on code-reducible tasks.
Case studies: consolidating long invoices without chunking/embedding, mining raw logs, optimizing an agent harness from its own traces, and generating a security report across 500k lines of code.
His bet: models post-trained to be RLM-aware will make this paradigm far stranger and more powerful.

More from AGI Musings