Pick the Right Question, Let AI Grind: Lessons from Math Proofs to Architecture Search
Soulren · reddit · 2026-08-19
The author's core observation: this year's landmark results — a model advancing Riemann Hypothesis work from 41.6% to 67.2% (verified in Lean, reviewed by outside mathematicians) and another solving a 27-year-old group theory question for roughly $2,000 in compute — weren't cases of AI choosing problems. Humans picked the target; AI did the exhaustive, checkable execution. This mirrors how the Transformer itself happened: the core of "Attention Is All You Need" was one statable hypothesis — recurrence isn't necessary for sequence modeling.
The essay cites the multi-agent system AIRA-Compose as a running version of this process: it searches attention/MLP/Mamba component combinations, tests candidates cheap and small, then scales winners — finding architectures (AIRAformer-D) that beat Llama 3.2 by 2.4–3.8% downstream. Caveats: wins are confirmed only at proxy scale, and most tested hypotheses come back as noise.
Actionable recipe: pick a narrow, falsifiable target stated as a clean hypothesis (vague prompting produced most noise); let the model recombine known components and verify ruthlessly, never trusting a single self-report; test cheap before big — identical small-scale controlled comparisons with one change, and treat small-scale wins as leads, not guarantees.
More from AGI Musings
- Brain Drain to 'Silicon Tower' Threatens Independent AI Research — sethlazar · 2026-08-19
- Global AI inference hits ~10 quadrillion tokens/month, overtaking humans next year — johnowhitaker · 2026-08-19
- AI projects are underrated: propose a thought model and iteratively fix it — GlenBradley · 2026-08-19
- Diagnosing 'BTHB-26': Satire on tech leaders' AI overhype — DrDatta_AIIMS · 2026-08-19
- Gary Marcus clarifies stance: Pure deep learning hits a wall — GaryMarcus · 2026-08-19
- Global inference may hit 10 quadrillion tokens a month, mostly unread — johnowhitaker · 2026-08-19