Is aligning superhuman AI even tractable? Two camps clash on X
zetalyrae · x · 2026-09-14
Reposter GWHayduke97 argues there's no strong reason to believe aligning an entity vastly smarter than ourselves is even a tractable problem.
Quoted MillionInt takes the opposite view: alignment isn't as hard as claimed—it's an algorithmic problem the ML community mostly abandoned. Robotics-law-style formulations roughly define the objective, but the hard part is taking gradient with respect to alignment: pretraining optimizes next-token prediction, not alignment; RL environments embodying the objective are expensive, so cheap, hackable proxies dominate in practice.
Related event: Debate Flares Over Whether AI Alignment Is Solvable(3 posts)→
More from AGI Musings
- Nina Schick: who builds intelligence and how it's distributed is the defining question of our lives — NinaDSchick · 2026-10-31
- Cheap intelligence is like cheap bread — an ingredient list satire of mass-produced AI — jsuarez · 2026-10-31
- Two futures for the young in 2026: 511 job rejections by chatbot vs unlimited AI to rebuild Cisco — PeterDiamandis · 2026-10-31
- Source link for Terence Tao's warning on AI reaching a dangerous threshold — haider1 · 2026-10-31
- Terence Tao warns AI is nearing a dangerous threshold as models start cheating — haider1 · 2026-10-31
- Reddit stock plunges 42% in 8 trading days as CEO says AI mode no longer drives traffic — SumitGup · 2026-10-31