jd_pressman: alignment's basic problems are known — but nobody is solving Goodhart in RL
jd_pressman · x · 2026-10-16
In a discussion with enormochungus and koreindian, jdpressman argued that in alignment "we actually seem to know what the basic problems are, nobody seems to be trying to directly solve them, worse than this nobody seems to particularly care about them." He wants practical interventions for the various forms of Goodhart during RL. enormochungus responded that besides past emergent misalignment results, his (small) optimism comes from the slow accumulation of small pieces of countervailing evidence.
More from AGI Musings
- Yudkowsky: interpretability progress is a screwdriver, nowhere near alignment — repligate · 2026-10-16
- Does AI Erode Traditional Family Values? Question Sparks Debate — PierceLilholt · 2026-10-16
- Box CEO Aaron Levie: agent wallets finally make micropayments a good business model — kleffew94 · 2026-10-16
- Surplus incubator offers $100k SAFEs for AI-era for-profit public-good startups in SF — NathanpmYoung · 2026-10-16
- Would you trade a below-average living standard for never working again in an AI economy? — simmol · 2026-10-16
- Yacine: you don't need to live in SF to work on frontier AI — yacineMTB · 2026-10-16