Telperion retrofits Qwen3.8-27B with training-free latent attention: 4x KV cache cut, +6.58% SWE
fulgencio_batista · reddit · 2026-10-15
Research group Telperion released training-free latent attention (MLA and GLA-g2) retrofits of Qwen3.8-27B that compress the KV cache by 4x.
Key numbers:
- The MLA variant scores 42.1% on SWE-bench Verified (n>1500) vs 39.5% for the base model, a +6.58% gain, while using 20%+ fewer reasoning tokens
- GLA-g2 keeps the full 4x compression at TP=2, addressing MLA's multi-GPU serving overhead
- FP8/INT4/EXL3 quants are offered: FP8 drops memory to 31.5GB with only 0.06% top-1 disagreement, beating same-size gguf quants in fidelity
- vLLM plugins provided, verified on RTX PRO 6000s, H100s, and 2x RTX 5060Ti 16GB
The method works on nearly any model using grouped query attention, requires zero retraining, and will be open sourced soon, with a benchmarked v2 and Qwen4-27B variants planned.
More from Models
- Opus 5.5 still leads a month later, but open-weight decision models are closing in — airesearch12 · 2026-10-16
- Google restricts Gemini Pro access for Google Plus subscribers, matching free tier — EmergencyLess8861 · 2026-10-16
- Meta releases Muse Spark Open, trained with vLLM-powered RL inference from day one — colinmcnamara · 2026-10-16
- Meta open-sources Muse Spark: runs on two Mac Studios, license bans passing output off as human-written — mawcopolow · 2026-10-16
- Golden Gate Claude explainer resurfaces: interpretability premise still holds — HarperSCarroll · 2026-10-16
- Hugging Face livestreams release of a new open-weight model — huggingface · 2026-10-16