Inkling 2's KV cache is 10x DeepSeek's, fueling doubts about RSI-driven architecture gains
bookwormengr · x · 2026-11-08
The author argues recursive self-improvement (RSI) is unlikely to drive architecture-level breakthroughs soon: AI excels at combining existing ideas through trial and error, but ideas are cheap while compute is expensive, so RSI mostly assists great human researchers rather than independently producing elegant architectures.
Evidence cited:
- KV cache size: Thinking Machines' Inkling 2 needs 8.2 GB per million tokens vs DeepSeek v4.1's 0.8 GB — roughly 10x worse. An attached estimate shows total KV cache of 32.85 GB at BF16 and 8.21 GB at FP4.
- Prefill cost: Inkling 2's 17B active parameters make prefill about twice as expensive as DeepSeek 4.1 Flash (8B active).
His hypothesis: most recent quality gains come from data, RL environments, and human-grounded goals — not from algorithmic self-improvement.
More from Infra
- Shruti open-weights speech-to-text covers all 22 Indian languages offline in under 30MB — kingslyj · 2026-11-08
- Dev runs SmolLM-135M inference entirely in SystemVerilog via Verilator — cneuralnetwork · 2026-11-08
- How Robert Dennard's 1966 One-Transistor Idea Became the Foundation of Modern DRAM — cneuralnetwork · 2026-11-08
- Anthropic API users beware: cache 5-minute window can make bundled credits hilariously wasteful — teortaxesTex · 2026-11-08
- Harbor turns one: 200T tokens/week, 10M+ Terminal-Bench runs, 1B+ rollouts — BenBlaiszik · 2026-11-08
- LLM-leaderboard-style atlas benchmarks silicon photonics foundries on component specs — jwt0625 · 2026-11-08