UMAP arcing explained: one giant coordinate dominates neural network activation maps
leland_mcinnes · x · 2026-10-27
Why do visualizations of neural network activations "come out as one big arc"? A new explanation:
- Vision models (e.g. ConvNeXt): a single coordinate takes values so much larger than everything else that "most similar" reduces to "closest in that one number." UMAP, which places each point next to those with most similar activations, ends up sorting image locations low-to-high along this giant coordinate—producing an arc while discarding all other structure. Zero it out and the map regains rich, clustered geometry.
- LLMs: the giant coordinate acts as an on/off switch—it takes enormous values on 1% of tokens (sink tokens, e.g. first tokens) and zero elsewhere. UMAP isolates the sinks into their own cluster and builds the rest of the map normally, which explains why LLM maps worked without a fix.
- The accompanying figure reproduces this on Llama-3.1-8B layer 16 (4,000 AG News tokens, Euclidean, 30 neighbours, mindist 0.08), contrasted with Cammarata's "Neural Networks Are Already Clean" figures.
More from Research
- OpenAI claims its models refuted the Exponential Time Hypothesis; proposers are checking — ZeroCool86 · 2026-10-27
- AI-designed drug works in humans: aging looks 'like a list of fixable problems' — rand_longevity · 2026-10-27
- Pessimistic complexity theory hypothesis: faster algorithms must get uglier — Elijah_Meeks · 2026-10-27
- NeurIPS benchmark: tree models still beat deep learning on 45 tabular datasets — tak3sh8 · 2026-10-27
- Kevin Bryan: maximizing raw accuracy is rarely the right way to train AI assistants — Afinetheorem · 2026-10-27
- Szegedy: a flaw in common training losses went unnoticed for over a decade — ChrSzegedy · 2026-10-27