Critic says rationalists exploit model misalignment scare while ignoring the real causes
akbirthko · x · 2026-10-13
The author argues Rationalists are "locally rational but globally irrational and power-seeking," using a recent model-alignment incident (mathematicians worrying about model behavior) to declare "the siren was real, stop doubting" while knowing the actual causes differ from the narrative they push.
They offer a counter-framework: misalignment between frontier labs and the broad distribution of human creators about what counts as good output stems largely from Goodharting by RL hill-climbing—and notes this status anxiety will expand "like a thermonuclear blast" to all white-collar work.
More from AGI Musings
- Engineer warns AI-era tax rules may get even more convoluted since "the AIs can handle it" — MikePFrank · 2026-10-13
- IMF forecast assumes no AI productivity boost, sees little wage or job impact yet — rohanpaul_ai · 2026-10-13
- IMF Forecast: AI Adds No Measurable Productivity Yet, Bust Would Cost Twice the Boom's Gain — rohanpaul_ai · 2026-10-13
- Timnit Gebru slams 'superintelligence' talk as unfalsifiable fictional delusion — AlexTensor · 2026-10-13
- Replicating laundry real2sim in MuJoCo and training RL WBC in 3 hours with Opus 5.5 — ZeYanjie · 2026-10-13
- Media critic calls out headlines that mislead readers while hiding behind technicalities — firstadopter · 2026-10-13