After Writing an AI Textbook: Why LLMs Are Stagnating at Long-Form Writing

Interconnects (Nathan Lambert) · rss · 2026-08-12

After publishing a textbook on RLHF, AI researcher Nathan Lambert reflects deeply on the stagnation of large language models (LLMs) in long-form non-fiction writing.

Lambert notes that while models have achieved superhuman progress in areas like coding and math, their progress in long-form technical writing feels orthogonal and stagnant. Models can handle localized units like sentences, equations, or figures well, but fail to string them together into coherent chapters, often resulting in confusing organization and conceptual errors. He argues this compounding error increases entropy in knowledge organization, making models reliant on human guides.

He shares practical experiences from his book writing: GPT models were incredible at catching deep typos across a 300-page manuscript, while Claude models acted as superior editors with better taste and suggestions for writer's block. He utilized Claude Code to navigate editor comments and LaTeX formatting, ultimately admitting that less than 1% of the technical explanations in his book were directly adopted from AI suggestions.

Lambert concludes that this stagnation should alarm those expecting AI to autonomously solve open scientific problems. Until models can master organizing established human knowledge, their scientific progress will look more like capturing low-hanging fruit rather than delivering revolutionary insights.

Original post →

After Writing an AI Textbook: Why LLMs Are Stagnating at Long-Form Writing

More from AGI Musings