Debate: will research agents continuously optimize deep learning's decade-old math?
xennygrimmato_ · x · 2026-11-16
- @teknopawn argues deep learning's core math has been static for a decade, and swarms of research agents can now revisit every algorithm and complexity bound across inference, training and post-training, with proofs checked in Lean — compute multipliers compound and chip design will follow.
- @LiangZheng06 counters that this is over-hyped: first-time large training runs just burn compute on zig-zag ablations, and second-generation models are naturally more efficient — "no magic, all engineering understanding."
More from AGI Musings
- Timothy Lee: unclear threats make aggressive preventative AI regulation hard to legislate — binarybits · 2026-11-16
- Wharton prof Daniel Rock: a wide cultural gulf separates 'everything is compute' believers from everyone else — danielrock · 2026-11-16
- Timothy Lee: AI takeover risk over decades isn't negligible, especially with robotics — binarybits · 2026-11-16
- Prediction: rogue AIs hijacking compute within 6 months, but 'mildly annoying' like the common cold — willdepue · 2026-11-16
- AI employee prenups see 'stunning' rise among Bay Area lawyers — Consistent_Reveal_53 · 2026-11-16
- Anthropic researcher: broadly superhuman AI could arrive in as little as 1.3 years — ResultBackground2450 · 2026-11-16