Stanford CS336 restarts with Percy Liang on BPE tokenization in the LLM lifecycle
stanfordnlp · x · 2026-10-16
Stanford NLP shared a clip from the restarted CS336 "Language Modeling from Scratch" course: Percy Liang explains how tokenizers convert raw bytes into integer token sequences (with an encode/decode demo on "Stanford was founded in 1885."), and walks through Byte-Pair Encoding—starting from single bytes and repeatedly merging the most frequent adjacent pairs, so "unhappiness" can split into "un"+"happi"+"ness". BPE endures because it balances vocabulary size against context length.
More from Research
- Embedding arithmetic demo: cartoon − animation − comic = diagram — mayfer · 2026-10-23
- Domingos Cites New Result Claiming RL Is 'Basically Useless' vs Supervised Learning — pmddomingos · 2026-10-23
- Wolfram Opens Rule 30 Prizes to AIs: $30,000 for Three Open Problems — cephaloform · 2026-10-23
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23