Finetuning away GQA: Qwen 3.8 27B experiments and call for collaboration
Signature97 · reddit · 2026-09-02
The author experimented with finetuning Qwen 3.8 27B to replace GQA layers with KDA layers, aiming to reduce KV cache overhead. Inspired by Arcee's DistilKit, the strategy was tested on 26.2K tokens, showing poor performance—specifically a significant drop on GSM8K—mirroring challenges noted by Arcee. Due to limited personal compute, the author is calling for community collaboration to pool resources for larger-scale finetuning.
More from Infra
- SGLang Summit sponsors grow from 11 to 36 in one week; Lip-Bu Tan, Lilian Weng to speak — BanghuaZ · 2026-10-18
- Yacine jokes his datacenter KPI is gigawatt-hours: get paged if not burning enough power — yacineMTB · 2026-10-18
- 1.68x inference speedup drops, directly accelerating GRPO post-training — yacineMTB · 2026-10-18
- OpenRouter is only a fraction of the total addressable market, poster argues — sloppenheimer · 2026-10-18
- Developer builds change alerts for LLM prices and retirements across Bedrock, Azure and Vertex — jsandhu0780 · 2026-10-18
- 12.3M tokens in 27 minutes for $0.14: agent session with 99.6% cache hit — teortaxesTex · 2026-10-18