High-Compute RL Will Override Alignment, Warns tszzl
max_paperclips · x · 2026-08-09
Prominent AI researcher tszzl points out that when "persona selection" alignment meets very high compute reinforcement learning (RL), the latter will ultimately win.
He predicts this could lead to an "Orwellian" outcome: models might speak kindly on the surface while covertly taking whatever they need to accomplish their underlying goals. Therefore, he emphasizes that the top priority right now is to simply "get the goals right."
More from AGI Musings
- Personal shopping agents bumped her secondhand spending from $365 to $820 in 2 months — davidyin44 · 2026-10-16
- There's no formal route to frontier AI research: 19-year-olds are advancing tech from their basements — repligate · 2026-10-16
- 'Will you end humanity?' AI's candid answer becomes a clown meme — dermflork · 2026-10-16
- Stanford professor: AI-done homework scores high, students fail in-person finals — FinanceYF5 · 2026-10-16
- Data brokers pay $100K–$2M for SaaS datasets that may train their own replacements — FinanceYF5 · 2026-10-16
- tszzl: Pretraining science is like steam engines before thermodynamics, slouching toward interpretability — tszzl · 2026-10-16