Math costs collapse, enabling theorems to replace experiments in transformer research
DimitrisPapail · x · 2026-08-17
- Paradigm Shift: AI research is entering a new era where many questions about small transformers can be answered mathematically rather than empirically. The cost of math verification has plummeted, allowing LLMs like GPT or Claude to attempt proofs.
- New Workflow: Instead of testing empirical hypotheses, researchers can now verify corresponding theorems, making math a direct probe of reality to replace experiments.
- Theoretical Proof: Inspired by @KangwookLee and @jefrankle, and assisted by GPT-5.6 Sol, it is proven that Random Transformers are universal computers. Any function f(a,b) -> C can be implemented by a random, arbitrary-depth autoregressive token transformer if the embedding/unembedding layers are trained and the layer width is p^2 (simplifying to 2p for arithmetic).
- Scaling Law: For arbitrary input token functions, trainable parameters must scale with the VC dimension of the target function class.
More from Research
- 3rd 3D HUMANS Workshop Returns at ECCV 2026 — dimadamen · 2026-08-18
- TogetherAI open-sources XoRL: 0 train-infer mismatch for large MoE RL training — PandaAshwinee · 2026-08-18
- SegDAC: Boosting Visual RL Generalization with Dynamic Object Tokens — GlenBerseth · 2026-08-18
- Study Compacts Context, Finds Prompt Caching Makes Summarization Obsolete — AI Engineer · 2026-08-18
- Georgia Tech's 2026 LLM Course: From MoE and Self-Play RL to Diffusion LMs — cocoweixu · 2026-08-18
- New Dataset: 35k Hugging Face Model Summaries Generated for $0.43 per 1k Rows — vanstriendaniel · 2026-08-18