Browser tool visualizes LLM attention token-by-token, surfacing the attention sink effect
Minute-Mountain2665 · reddit · 2026-10-12
A developer built Attention Galaxy, a browser-only visualization tool running SmolLM2-135M locally via onnxruntime-web (int8, no server, works on phones).
- Generates one token at a time, showing next-token candidates with probabilities
- Draws attention from the current token back to earlier ones as labeled weighted links; click any word to see where its attention went, ranked
- Includes a layer slider and a toggle to hide attention sinks
The author's key takeaway: heavy attention on the first token looked like a bug at first, but turned out to be the well-known attention sink phenomenon. Open-source with a live demo.
More from coding & agent
- lightningprox-mcp offers pay-per-request Claude/GPT access via Bitcoin Lightning, no API keys — modelcontextprotocol · 2026-10-12
- Hive Morph ships polymorphic identity and capability tokens for autonomous agents — modelcontextprotocol · 2026-10-12
- Subagents: why more agents can cost less — Opus orchestrating Haiku workers — RelevantEmergency707 · 2026-10-12
- Replacing LLM calls with deterministic code cuts compliance alert costs from $290K to under $26K — josephdviviano · 2026-10-12
- Open-source MCP image/video server: Qwen-Image 2.1 Turbo runs on 4GB VRAM with 16 tools — Delicious-Farmer-234 · 2026-10-12
- OpenCode 2.0.26 users report agents stalling with "Great, I'll implement that" — needing a nudge — SM8085 · 2026-10-12