DeepMind's SkillSmith: Composing Parametric Weights and Textual Knowledge at Inference Time
rohanpaul_ai · x · 2026-08-03
Google DeepMind's new paper introduces SkillSmith, a novel method to dynamically compose model weights (parametric skills) with textual knowledge.
- Mechanism: Treats prefix key-value (K-V) caches as an input modality. For new tasks, it interleaves existing prefix weights with task descriptions, examples, and rationales.
- Inference-time Initialization: A forward pass generates a fresh prefix cache for a frozen Gemma 3 4B model, enabling capability composition at inference time rather than requiring target-specific training.
- Results: On the Composite-SNI benchmark, combining both K-V caches and text yields a significantly higher Elo score (1,714) than using either alone. The generated prefixes also provide a stronger starting point for downstream fine-tuning, especially on difficult, data-sparse tasks.
More from Research
- Simplex Diffusion Models paper talk by EPFL/DeepMind's Justin Deschenaux on Oct 19 — ArnaudDoucet1 · 2026-10-16
- Meituan LongCat team's Data World synthesizes data environments to train data agents — meituan-longcat · 2026-10-16
- FDE-Gym benchmark shows huge gap on real AI delivery tasks: top model scores 0.563 — DeepLethe · 2026-10-16
- ForcingBooster halves computation of autoregressive video diffusion with patch-wise denoising — VAST-AI · 2026-10-16
- Baidu open-sources FlashMask: up to 1.55x faster than FlashAttention-4 for long-context training — BaiduResearch · 2026-10-16
- SemEval-2027 Task 9 opens: multimodal news framing analysis across 4 languages — preslav_nakov · 2026-10-16