Agent-Generated Heatmap Shows Inkling 2 Small's Split Q/K/V Queries Diverge in Later Layers
stochasticchasm · x · 2026-11-13
The author had an agent plot cosine similarities of all 112 learned depth queries in Thinking Machines' Inkling 2 Small checkpoint. The heatmap shows split q/k/v stream queries are much more dissimilar to each other—especially in later layers—than regular attention-residual queries. Inkling 2 Small adds attention residuals with q/k/v split, gated attention, dynamic attention sink, dynamic shortconv, and value norm over the previous generation.
Related event: Heatmap Analysis Exposes Attention Splitting in Inkling 2 Small(2 posts)→
More from coding & agent
- Dev builds an OP-1-inspired skill for agentic coding, posted on GitHub Next — narphorium · 2026-11-13
- vork: open-source headless 3D/VFX engine for AI agents, renders black hole bending light in 18 min — ravisparikh · 2026-11-13
- Agent swarm spends $3M/month in tokens to build game Spawn, treats an hour without shipping as SEV-0 — dl_weekly · 2026-11-13
- GitHub Copilot CLI v1.0.110 adds masked sudo password prompts for agents and credit-aware /compact — copilot-cli-release-app[bot] · 2026-11-13
- Replit Agent can now buy supplies via Link and Visa and run Google Ads for your business — amasad · 2026-11-13
- 2-bit DeepSeek V4.1 Flash runs a coding-agent seat on two 2016 V100s: a 12-hour real-work trial — a4mula · 2026-11-13