'Zombie ideas': many still believe most LLM capability comes from pretraining, says AI explainer author
batterystaple42 · x · 2026-11-03
In an X exchange, someone notes many people still believe most LLM capabilities come from pretraining. Understanding AI's Timothy B. Lee calls this a 'zombie idea' held by those who learned about LLMs in 2023-2024 and haven't followed developments since — implying recent capability gains come mostly from post-training/RL.
Related event: Tim Lee: Post-Training, Not Pre-Training, Drives LLM Capability(3 posts)→