Kording Lab Explains Modern LLMs: Old Recipe Plus Inner Speech, Tools and Agents

On August 26, the University of Pennsylvania's Kording Lab published a series of posts systematically breaking down where modern large language models get their capabilities. The core argument: today's LLMs are not a leap out of nowhere but an extension of the old recipe—building on traditional "next-token prediction" neural network training while layering in latent inner language, tool calling, agent coordination mechanisms, and better evaluation, which together constitute today's AI capabilities.

Confirmed

• @KordingLab notes that older LLMs, exemplified by 2022-era models, could produce impressive text but were essentially "stochastic parrots," often remaining superficial; AI has since moved far beyond that stage at remarkable speed.
• The first improvement is "latent inner language": having the model generate text users cannot see as a scratchpad or working memory, using internal deliberation to help produce higher-quality final answers.
• The second is tool calling: letting the model invoke external programs such as databases, web search, and calculators to compensate for gaps in its own knowledge or computation.
• The third is building agents: combining componentized LLMs, memory modules, databases, and tool-calling capabilities, coordinating the components to jointly solve complex tasks.
• Two further strategies are proposed for raising output quality: in domains with a defined "ground truth" (e.g., math, coding), push models toward that truth via training objectives; in open-ended domains without clear ground truth, use LLM self-evaluation to check answers, going beyond simple probabilistic generation.

Why it matters

• The series offers a unified component-based framework for understanding the strengths and weaknesses of today's LLMs: the modern interactive "thinking" machine is a combination of the old recipe plus multiple component innovations, not a single technological breakthrough.
• For researchers and practitioners, the progressive path of "inner language — tool calling — agents," along with the distinction between "truth-guided vs. self-evaluated" scenarios, points to a clear direction for improving LLM output quality.

2026-08-26 ~ 2026-08-26 · 7 related posts

• From 'Stochastic Parrots' to Rapidly Evolving Modern LLMs — KordingLab · 2026-08-26
• Improving LLM Answers: The First Idea, Inner Speech — KordingLab · 2026-08-26
• Enhancing LLM Answers: The Second Idea, Tool Use — KordingLab · 2026-08-26
• Generating Answers: The Third Idea, Agents — KordingLab · 2026-08-26
• [source] Improving LLMs: Steering Towards Truth and Self-Evaluation — KordingLab · 2026-08-26
• [source] The Evolution of Modern Interactively 'Thinking' Machines — KordingLab · 2026-08-26
• [source] Today's LLM Strengths and Weaknesses Trace Back to Component Ideas — KordingLab · 2026-08-26