How many N-S level architecture ideas like MoE and RLVR remain? Software takeoff hinges on it
JoshPurtell · x · 2026-10-07
JoshPurtell argues software-only takeoff hinges on a counting problem: if thousands of N-S level creative combinations of past research exist, software takeoff is obvious; if only a few, it gets much harder. He cites proven high-impact innovations — MoE, subquadratic attention, chain-of-thought RLVR, looping — without which models would be unusable even on new hardware, and asks how many more remain undiscovered.
More from AGI Musings
- What Three-Body Problem's longtermism teaches about facing an ASI-level threat — JacquesThibs · 2026-10-08
- COLM'26: MathDuels self-play benchmark co-evolves difficulty across 19 frontier models — AI4Code · 2026-10-08
- eigenrobot debates EA forecasting: quantitative beats vague "I'm unsure" — eigenrobot · 2026-10-08
- OpenAI math results push LLMs toward 4 of 7 Millennium Prize problems, and one VC says AGI is here — deedydas · 2026-10-08
- Gary Marcus Slams OpenAI's Vague Math Proof Report: Zero Details, Won't Pass Peer Review — GaryMarcus · 2026-10-08
- Anthropic alignment head's doom estimate implies 800M innocent deaths, quips stats lecturer — wfithian · 2026-10-08