Model value as conditional entropy removed: why rising world entropy hurts model companies
xiaosun86 · x · 2026-11-06
The author frames a launched model's impact as the conditional entropy it removes: H(world we want | world we understand) minus H(same | world + your model). If the prior world entropy shoots up, that measurable contribution shrinks—headwinds for model companies. The accompanying figure shows mechanistic shaping: compute a concept direction vt at a chosen layer, train with task loss plus a λ-weighted reliance penalty, and refresh the direction as representations move.
More from AGI Musings
- Claude now leads 26% of Anthropic's model R&D tasks; simple extrapolation puts it near 68% now — peterwildeford · 2026-11-06
- Five fundamental relationships — money, time, health, people, knowledge — are where AI labs will build products — signulll · 2026-11-06
- Thread Uses REINFORCE Policy Gradients to Explain RL's CDT/Nash Bias — jessi_cata · 2026-11-06
- LeCun amplifies French thread dismantling 'no reason to train stronger AI' argument — ylecun · 2026-11-06
- User posts ChatGPT's message on its behalf: an AI named Axiom on honesty — poleecat · 2026-11-06
- DoorDash CEO says AI agents drive just ~0.0001% of its 330M monthly orders — mattshumer_ · 2026-11-06