Visualizing Claude Opus 4.6's Internal Multiplication: Two Years of Experiments Become a Paper
While finishing a paper, ctjlewis posted a series of visualizations of Transformer internals, using Claude Opus 4.6 multiplying two 128-digit integers as a case study, with animated trace diagrams showing the model's internal computation token by token. The work covers two years of computational experiments across several model generations, and the author believes the model's multiplication algorithm is taking shape.
Confirmed
- The visualization method marks exactly which token is written at each step, color-codes operation slots, and shows how information flows from one token to the next; solid fills denote computed values, while bordered cells denote copies from tape-memory.
- Training details: only 15 execution traces were used as examples, with a maximum size of 48×48; the model generalized from length 48 to 128 — a considerable leap given O(n²) complexity.
- The author's thesis: correctly answering hard questions is not just "saying the right token," but performing intermediate computation in the weights first and then emitting an answer in a narrow correct distribution, similar to how humans do mental arithmetic.
- The author showed the general form of the multiplication algorithm; the paper will cover two years of experiments across generations of models.
Why it matters
- In related discussion, some argued "we do understand how LLMs work"; this work offers concrete visual evidence for that view, tracking what the model does token by token rather than speculating.
- The author criticized academia, arguing such constructive exploration is more valuable than 99% of literature that merely complains about what models cannot do — a stance that itself draws attention to interpretability research directions.
2026-09-28 ~ 2026-09-28 · 6 related posts
Primary sources
- [source] Animated trace shows Opus 4.6 multiplying two 128-digit integers — ctjlewis · 2026-09-28
- Two years of experiments on model arithmetic beat most academic literature, author says — ctjlewis · 2026-09-28
- Visualizing every token and memory copy in transformer ops from 2 years of experiments — ctjlewis · 2026-09-28
- [source] Visualizing transformer execution traces: computed values vs copy ops from tape-memory — ctjlewis · 2026-09-28
- [source] 15 traces to generalize from 48 to 128: O(n^2) length generalization details revealed — ctjlewis · 2026-09-28
- 48 execution traces give length generalization from 48 to 128 as computation happens in the weights — ctjlewis · 2026-09-28