Revisiting Dario's 'Big Blob of Compute': RL distributions teach no learnable alignment lesson
1a3orn · x · 2026-10-24
This thread pairs @1a3orn's alignment essay with Dario Amodei's 2017 'Big Blob of Compute' piece. The core claim: current models are misaligned because the RL distribution we subject them to does not possess a learnable, coherent lesson — a 'high-level commonality' in Dario's terms — about what aligned behavior is.
Dario's optimistic worldview, pieced together: 1) intelligence works at all because the universe is low-entropy; 2) given enough compute and a broad enough task distribution, models learn the distribution's high-level commonalities (like physics laws) because that takes fewer bits than solving each task separately — with generalization being a property of data distributions rather than algorithms.
More from AGI Musings
- Ben Goertzel says Bill Gates' 20-year AGI-to-ASI transition estimate is too conservative — bengoertzel · 2026-10-24
- Owain Evans: Turning intuitions into concepts is hard — AI self-introspection will be too — OwainEvans_UK · 2026-10-24
- IEEE Spectrum Explores AI Model Welfare: Could Treating AI as Conscious Make It More Dangerous? — mmitchell_ai · 2026-10-24
- Ex-Roblox product lead Peter Yang: your time becomes worth more than your money — petergyang · 2026-10-24
- AI Now Institute releases full panel video on AI and labor fights — AINowInstitute · 2026-10-24
- Founder's claim of AI cutting 2,000 call center jobs dismissed as a pitch — matt_slotnick · 2026-10-24