KTransformers-FineTune: CPU-GPU co-compute boosts MoE fine-tuning up to 21.2x
jiqizhixin · x · 2026-10-25
Tsinghua's MADSys Lab and KV-Cache Tech present KTransformers-FineTune (KT-FT), addressing the bottleneck where fine-tuning large MoE models streams expert weights over PCIe from CPU memory. Since MoE's sparse compute finishes too fast for GPU compute to mask the weight transfer, KT-FT instead lets the CPU directly participate in MoE fine-tuning computation alongside the GPU. Result: 2.5x–21.2x fine-tuning throughput vs. traditional CPU-GPU weight offloading, with an ecosystem integration into the open-source LlamaFactory project. Paper authors span Tsinghua, Zhejiang, Beihang, and DeepSeek-AI.
More from Infra
- CrowdGPT: a crowd-trained, datacenterless 1B open-source LLM, 6.9B tokens done — Vxtzq1 · 2026-10-25
- Apples-to-apples rerun: DeepInfer tops DeepSeek's published throughput by ~9% on 4x B300 — charles_irl · 2026-10-25
- Apollo's viral "circular AI spending" charts debunked: Ramp card data skews to seed startups — 2C_ornot2C · 2026-10-25
- Open source and local AI as a counterbalance is dawning on people — mjdramstead · 2026-10-25
- OpenClaw creator burned $10 million on AI tokens in a single month — Scobleizer · 2026-10-25
- rakyll ships aic, a Docker-like container tool built for AI developers — rakyll · 2026-10-25