LLMs Are Mode-Covering, Brains Are Mode-Seeking: The Path to Post-Slop
beffjezos · x · 2026-08-31
Comparison of Mechanisms:
- LLM Training Objective: Minimizes cross-entropy, leading to "mode-covering" behavior (covering all possible probabilities), which often results in average or "slop" output.
- Biological Brain Mechanism: Minimizes variational free energy, leading to "mode-seeking" behavior (actively exploring and confirming specific states).
Conclusion: The path to a "post-slop" world lies in stochastic-thermodynamic algorithms and computation, rather than relying solely on current LLM training paradigms.
Related event: AI Mode-Covering vs Brain Mode-Seeking Sparks Debate(2 posts)→
More from AGI Musings
- AI risks rapidly realizing; slow timelines are the real fantasy — davidmanheim · 2026-08-31
- Materialists Should Not Reject LLM Consciousness Entirely — granawkins · 2026-08-31
- DHH on Agent state: exponential change wipes out priors — Dan_Jeffries1 · 2026-08-31
- Guardian: Gen Z Doesn't Care Where Money Comes From, Embracing AI Training and OnlyFans — nordicinst · 2026-08-31
- Opinion: AI must shift from tech demos to products — gorkem · 2026-08-31
- Oxford Lab Event to Explore the Emerging Economy of Continually Learning AI Agents — MihaelaVDS · 2026-08-31