Thinking Machines open-sources looped model variant matching mini on coding benchmarks
SonglinYang4 · x · 2026-11-23
Songlin Yang of Thinking Machines released TML-d24-varloop, a research variant (on Hugging Face) of their looped architecture that reportedly matches or beats mini on coding tasks.
The discussion started from @bjoernpl's observation that the model doesn't share KV cache across looped layers, costing 32KB per token vs 16KB with full sharing — meaning far more memory for long contexts than models like Qwen. He suggested a hybrid approach: share the global KV cache while giving each looped layer its own local SWA cache, which reportedly doesn't degrade performance, and asked about ablations since this matters a lot for local inference.
Yang confirmed a terminal KV-sharing variant exists — the newly released model — performing on par with or better than mini on coding.
More from Infra
- Inference practitioner on GPU co-ops: saturation math, weight caching, and quantization as dark art — sloppenheimer · 2026-11-23
- Nvidia's tale of two weeks: record revenue alongside regulatory probes and governance drama — emmanuelvivier · 2026-11-23
- Oracle raises $20B ($10B equity, $10B bonds) to fund OpenAI infrastructure deal — emmanuelvivier · 2026-11-23
- DeepSeek on 4 DGX Sparks codes great with Opus 5.5 planning, GLM-5.3 not ready — AIFlow_ML · 2026-11-23
- A beginner's guide to running local AI with Inkling, Qwen, and Unsloth — Retell · 2026-11-23
- Micron CEO: 2027 supply 75% sold out, talks already moving to 2028 — Beth_Kindig · 2026-11-23