Deep learning is brute force: activation functions can't encode epistemic structure, argues ryunuck
ryunuck · x · 2026-09-21
The author argues there are infinitely many other ways to model information, each with different trade-offs and introspection probe qualities. Replying to a critic, he contends it can't be done because the substrate is activation functions and weights: knowledge is distributed, not localized, and the model is geometrically unstructured with respect to the geometry of epistemics — deep learning is brute force mixed with spray & pray.
Related event: Debate: Can LLMs Have Real Cognitive Structures?(2 posts)→
More from Research
- Founder says frontier lab's Jev copies his 2025 non-autoregressive decision model paper and open weights — abhijithneil · 2026-09-21
- Schmidhuber: LeCun's JEPA is essentially his 1992 Predictability Maximization system — SchmidhuberAI · 2026-09-21
- Score Centering is secretly a STE: new fix targets LLM RL training instability — brandondamos · 2026-09-21
- Most Researchers Still Treat Transformers as Black Boxes, and Public Understanding Is Decades Away — gerardsans · 2026-09-21
- Paper pinpoints EOS token mismatch as the root failure mode of on-policy distillation — tw_killian · 2026-09-21
- AI Haiku Study: GPT-5 and Gemini 2.5 Poems Indistinguishable From Human Work — s_scardapane · 2026-09-21