DeepMind's SkillSmith: Composing Parametric Weights and Textual Knowledge at Inference Time
rohanpaul_ai · x · 2026-08-03
Google DeepMind's new paper introduces SkillSmith, a novel method to dynamically compose model weights (parametric skills) with textual knowledge.
- Mechanism: Treats prefix key-value (K-V) caches as an input modality. For new tasks, it interleaves existing prefix weights with task descriptions, examples, and rationales.
- Inference-time Initialization: A forward pass generates a fresh prefix cache for a frozen Gemma 3 4B model, enabling capability composition at inference time rather than requiring target-specific training.
- Results: On the Composite-SNI benchmark, combining both K-V caches and text yields a significantly higher Elo score (1,714) than using either alone. The generated prefixes also provide a stronger starting point for downstream fine-tuning, especially on difficult, data-sparse tasks.
More from Research
- Netflix details its production LLM judge: hundreds of thousands of recommendations scored weekly — omarsar0 · 2026-08-24
- Nature Comment: Provenance, not interpretability, grounds trust in autonomous science — gabepgomes · 2026-08-24
- New Architecture RHEA: Train 1B Model on 8GB VRAM — zemondza · 2026-08-24
- Trained two 16M-param models to do generative CAD with real physics — debreuil · 2026-08-24
- Claude model helps discover complex structure on S^6, solving 60-year-old math problem — Singularitarian · 2026-08-24
- Study: Agents read instructions/notes 60.5% of the time, rarely touch API docs — dair_ai · 2026-08-24