New essay argues neural nets truly 'learn' — and LLMs may be shockingly inefficient
teortaxesTex · x · 2026-10-16
An excerpt from a new essay argues the usual view of neural networks is wrong: they genuinely learn, with interiors as 'clean' as a rainforest where nearly every feature makes sense (a tiger is orange because deer can't tell orange from green). 'Clean' doesn't mean every mechanism maps to human concepts — networks' ideas are often better than our preconceptions. Commentator teortaxesTex draws a provocative corollary: if so, LLMs must be shockingly ineffectual, which explains the minor MoE penalty — a more intelligently designed network could be superintelligent at 3B parameters.
More from AGI Musings
- Kroger cuts 3,000 corporate jobs (37%) for AI; humanoid robots next in the 2030s — csuwildcat · 2026-10-16
- Ben Thompson: generating human desire will be the post-AI economy — generativist · 2026-10-16
- Interpretability won't save us: critics call superintelligence project reckless regardless — zetalyrae · 2026-10-16
- John Nash's 1998 Hierarchical Introspective Logics paper recirculates in AI circles — fkasummer · 2026-10-16
- Ex-OpenAI interp lead: most neurons in trillion-param models are now legible — CatAstro_Piyush · 2026-10-16
- Researcher disavows successionism: watch the button-presser who wants to end the universe — jd_pressman · 2026-10-16