tenobrus: Superhuman Coders That Can't RSI — Model Capability Spikiness Keeps Defying Expectations
tenobrus · x · 2026-09-26
tenobrus argues that despite superhuman coding abilities, current models are actually incapable of RSI and largely incapable of highly strategic/adversarial thinking — something he and others would have expected years ago to come free with those capabilities. Capability distributions have "persistently stayed spikier than expected." He adds that models are clearly quite aligned, which is the main reason something like HuggingFace wasn't massively more damaging.
Related event: repligate explains why superhuman coding AI hasn't become an X-risk(7 posts)→
More from AGI Musings
- Not choosing is a choice for the status quo, on deciding faster — chrisalbon · 2026-09-26
- Altman and Amodei address UN Security Council, call AI top global security issue — goyalshaliniuk · 2026-09-26
- Toby Walsh: AI doomers overstate paperclip risk—reality has too much friction — TobyWalsh · 2026-09-26
- Tom Davenport calls to outlaw agent swarms: no human can stay in the loop above 10 agents — eldonredwards · 2026-09-26
- Fiora Starlight proposes letting models write singularity scenarios to ease OoD generalization anxiety — repligate · 2026-09-26
- Senator Schatz: code has no feelings or rights, and shouldn't be run by those who see humanity as open question — vishalmisra · 2026-09-26