Visualizing every token and memory copy in transformer ops from 2 years of experiments
ctjlewis · x · 2026-09-28
ctjlewis showcases a visualization of transformer internals that reveals the exact tokens written, color-coding each slot in the operation and tracing how information flows from previous tokens to the next one. Solid fills mark computed values; bordered slots are copy operations reading prior values from tape-memory.
The work will cover two years of computational experiments across several generations of models, and the author argues it beats 99% of academic literature, which he says is largely complaints about what models supposedly cannot do.
More from Research
- Continual-ARC: lifelong ARC-AGI stream finds agent harness persistence rarely beats simple caching — unifyai · 2026-11-16
- DySENSE: MIT's GenAI framework synthesizes satellite imagery with dynamic city energy maps — MIT · 2026-11-16
- WebArena-Pro: a 300-task multimodal benchmark across 20 self-hosted apps for web agents — McGill-NLP · 2026-11-16
- Training-free page-sparse decode in llama.cpp: 1.72× faster 128k decoding on an RTX 5090 — topppggg · 2026-11-16
- Training Qwen3-8B to drive a simulated hand spawns a dedicated internal 'pressure' direction — repligate · 2026-11-16
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16