Interactive lessons record a 1.5B model's attention and token probabilities live

techartist_ · x · 2026-11-20

The author ran a 1.5B-parameter open model locally on a Mac and recorded its full generation behavior: next-token probability distributions, which earlier tokens each word attends to, and guesses across all 28 layers, turning the recordings into interactive lessons.

Lesson 1 runs a real production tokenizer in the browser: "strawberry" tokenizes as a single piece — a key reason LLMs miscount its r's.

Related event: Developer Runs 1.5B Model Locally on Mac, Visualizing Attention Layer by Layer(2 posts)→

Original post →

More from Research

Research channel →