RLM Podcast: Recursive calls may outperform long context scaling

lateinteraction · x · 2026-08-30

This Weaviate Podcast episode features MIT PhD student Alex Zhang discussing Recursive Language Models (RLMs), a new abstraction for agent harnesses. Instead of stuffing tool observations into an ever-growing prompt, RLMs treat the prompt as a variable; the model writes code to manipulate its own context and spawns recursive LLM calls on pieces of it.

Key insights:

Long context limits: Stacking tool outputs into one giant sequence is terribly out-of-distribution. Labs spend massive resources to make these trajectories in-distribution.
RLM advantage: Inspired by DSPy, it lets models decompose tasks themselves, relieving context pressure so each call handles a small, local, in-distribution problem.
PrimeAgent practice: Prime Intellect's production harness built on RLMs uses a persistent IPython REPL as its only tool. It achieves strong results on ARC-AGI-3 and long-horizon benchmarks with significant token savings.
Speculative tool calling: A preview of a new technique borrowing from speculative decoding. A shadow executor queues sub-agent calls while the main model generates code, delivering 2x speedups.

Original post →

RLM Podcast: Recursive calls may outperform long context scaling

More from coding & agent