Agent-plotted heatmaps reveal split-stream queries diverge sharply in later layers
stochasticchasm · x · 2026-11-13
The author had an agent plot cosine-similarity heatmaps of attnres queries plus split q/k/v stream queries, finding that split-stream queries are notably dissimilar from each other and from normal attnres queries specifically in later layers. MLP and attention queries also stay consistently different.
Related event: Heatmap Analysis Exposes Attention Splitting in Inkling 2 Small(2 posts)→
More from Research
- Incremental research is now riskier than going all in on big ideas — fkasummer · 2026-11-13
- MIT & Meta paper: ReHarness revises agent memory notes with later outcomes, beats baselines — rohanpaul_ai · 2026-11-13
- MASCOT (EMNLP 2026) trains multi-agent role-played dialogue, hitting 80.0 persona consistency — jiqizhixin · 2026-11-13
- Stanford's CS312 'Deep Learning Alchemy' opens quizzes and materials to the public — stanfordnlp · 2026-11-13
- Blogger Works Through Stanford CS31: Residuals Matter More Than Normalization — stanfordnlp · 2026-11-13
- Stanford Course Shares Data: AI Users Score Higher on Homework, Worse on Exams — stanfordnlp · 2026-11-13