roon: no fundamental alignment progress since RLHF, capabilities racing far ahead
theteknosaur · x · 2026-11-17
OpenAI researcher roon (@tszzl) posts (screenshot attached): there have been no fundamental alignment advancements since RLHF — most work remains folk theory; the prediction that capabilities would race far ahead of interp and alignment has come true. Lab researchers privately admit alignment isn't solved, so racing into RSI is an unprecedented folly and frontier research must slow significantly soon. Reward shaping and monitorability matter, but a deeper layer under the behaviorist abstraction remains poorly understood. @theteknosaur adds: reward shaping teaches what to do, not whether it matters — capability with an internalized sense of its own harm is the alignment we haven't found.
More from AGI Musings
- Anthropic AI helped design CAR-T signaling domains 70% shorter with ~10x receptor improvement — ChrisGPT · 2026-11-17
- Anthropic Helped Engineer CAR-T Cells With 70% Shorter Signaling Domain, 10x Better Expression — ChrisGPT · 2026-11-17
- Slotnick hopes AI-augmented society needs more historians and philosophers, not fewer — matt_slotnick · 2026-11-17
- Figure 04 passes 'robotics Turing test': gloved robot hand indistinguishable from human — PeterBowdenLive · 2026-11-17
- We're giving machines the ability to reason while losing the habit of thinking — OwariDa · 2026-11-17
- Top AI researcher: embedding audits by METR and Apollo are priced in, mechinterp is the exciting frontier — tszzl · 2026-11-17