Paper frames Transformers as discretization of continuous integro-differential equations
NandoDF · x · 2026-09-13
新论文《A Mathematical Explanation of Transformers》为 Transformer 与大语言模型架构构建了严格的数学框架：把 Transformer 视为连续积分-微分方程的离散化，自注意力对应非局部积分算子，层归一化对应到约束集合上的投影，前馈层与激活函数纳入同一框架；再用算子分裂和数值离散化还原标准 Transformer 架构，并推广到多头注意力、Vision Transformer 和卷积 Transformer。
More from Research
- Terence Tao's new essay: AI shifts math's scarce resource from finding proofs to understanding them — NandoDF · 2026-09-13
- Raji calls for a new edition of 'Who Audits the Auditors?' for frontier AI — rajiinio · 2026-09-13
- Researcher Calls for Better Methods for Frontier AI Oversight, Lists Missing Topics — timrudner · 2026-09-13
- NVIDIA breaks down the open Nemotron post-training pipeline: Data Designer, Gym, and RL — NVIDIA Developer · 2026-09-13
- Scholar's €750 Code Bug Bounty Now Gets Multiple AI-Found Bug Reports Daily — Singularitarian · 2026-09-13
- Recurrent Looped Transformer (RLT) open-sourced: latent reasoning with infinite depth — inductionheads · 2026-09-13