Interactive lessons record a 1.5B model's attention and token probabilities live
techartist_ · x · 2026-11-20
The author ran a 1.5B-parameter open model locally on a Mac and recorded its full generation behavior: next-token probability distributions, which earlier tokens each word attends to, and guesses across all 28 layers, turning the recordings into interactive lessons.
Lesson 1 runs a real production tokenizer in the browser: "strawberry" tokenizes as a single piece — a key reason LLMs miscount its r's.
More from Research
- Murmur Transcribes Long Audio 2.5x Faster Than Single-Pass, Beats WhisperX — bariskasikci · 2026-11-20
- AI-designed CPU 'Frostbite' claims 4.59x perf-per-watt over silicon-proven design — davidcrawshaw · 2026-11-20
- OpenAI data: agents reshape research but experiments-per-researcher up only 1.68x — CatAstro_Piyush · 2026-11-20
- Salesforce releases BLIP-Live: SOTA proactive streaming video understanding with PAUC-F metric — Salesforce · 2026-11-20
- RICoT paper: LLMs read page streams like humans for document segmentation, boundary F1 0.915 — Prashant Khatri · 2026-11-20
- One neuron per language hits 99.1% language-ID accuracy, beating dedicated models — julianweisser · 2026-11-20