Researcher: alignment's basic problems are known, yet almost nobody works on them directly
jd_pressman · x · 2026-10-16
AI researcher JD Pressman argues that "we actually seem to know what the basic problems are, nobody seems to be trying to directly solve them, and nobody seems to particularly care." He calls for practical interventions against the various forms of Goodhart's law during RL. His interlocutor notes that aside from past emergent misalignment results, few examples come to mind, and that their slim optimism rests on the slow accumulation of small pieces of countervailing evidence.
Related event: Alignment Researcher: Core Problems Known but Almost Nobody Solving Them(3 posts)→
More from AGI Musings
- Blogger predicts recursive self-improvement arrives around November 2027 — imjustnewatai · 2026-10-16
- 31 Tech Curves Visualized: Kurzweil's $1,000-Brain-Compute Call Missed by 400x — cpaik · 2026-10-16
- DeepMind Paper: First-Price Auctions in Agentic Markets Reach 1-1/e^2 Efficiency — dair_ai · 2026-10-16
- Garrison Lovely's new book 'Obsolete' argues AGI is neither impossible nor inevitable — GarrisonLovely · 2026-10-16
- Majmudar clarifies: controllability may be solved in 1-2 years, not full alignment — MajmudarAdam · 2026-10-16
- AI-generated fake Feynman lectures are flooding the internet, and they're bad — dreamwieber · 2026-10-16