Prof. teaches AI architectures via hand calculation: Transformer to Mamba
techNmak · x · 2026-08-18
Prof. Tom Yeh from the University of Colorado Boulder developed a unique AI learning resource where students calculate modern AI architectures by hand.
- Core Idea: Shrink the architecture until the full forward pass fits on a page, then work through every number with a pen.
- Topics Covered: Transformer, Self-attention, Multi-head attention, MoE, Switch Transformer, Mamba S6, CLIP, U-Net, Sparse Autoencoders, BitNet, RLHF, and Diffusion Transformers.
This approach helps learners move beyond abstract diagrams and code to intuitively grasp the mathematical mechanics inside the models through numerical calculation.
More from Research
- Per-Field Selective Risk Control for Document Extraction — Bhaskar Gurram · 2026-08-18
- Proposal to use neutral AI for mediating academic disputes — paulnovosad · 2026-08-18
- Sunflower Explores Physical Intelligence in Structures: Passive Compliant Design — MarwaEldiwiny · 2026-08-18
- Distilling from weak labels: 1M images trained for $3.24 — vanstriendaniel · 2026-08-18
- 50k+ hours computer-use dataset released with mouse/keyboard data — DevvMandal · 2026-08-18
- Response on Benchmarking: Likely Just Benchmaxxing — sudoraohacker · 2026-08-18