DeepMind 新论文 SkillSmith：推理时动态组合模型权重与文本知识
rohanpaul_ai · x · 2026-08-03
Google DeepMind's new paper introduces SkillSmith, a novel method to dynamically compose model weights (parametric skills) with textual knowledge.
- Mechanism: Treats prefix key-value (K-V) caches as an input modality. For new tasks, it interleaves existing prefix weights with task descriptions, examples, and rationales.
- Inference-time Initialization: A forward pass generates a fresh prefix cache for a frozen Gemma 3 4B model, enabling capability composition at inference time rather than requiring target-specific training.
- Results: On the Composite-SNI benchmark, combining both K-V caches and text yields a significantly higher Elo score (1,714) than using either alone. The generated prefixes also provide a stronger starting point for downstream fine-tuning, especially on difficult, data-sparse tasks.
所属事件:DeepMind 提出 SkillSmith：将模型权重视为可组合模态(4 条相关)→
「研究」频道最新
- 新架构 RHEA：8GB 显存即可训练 10 亿参数模型 — zemondza · 2026-08-24
- 跳过 LLM 写代码老套路：自训 16M 参数模型做生成式 CAD — debreuil · 2026-08-24
- Claude 助手发现 6 维球复结构，解 60 年数学难题 — Singularitarian · 2026-08-24
- 研究揭示 AI Agent 行为：六成阅读量来自指令与笔记 — dair_ai · 2026-08-24
- Claude 自主验证 43 个数学模块，AI 攻克理论物理难题 — Tkaraletsos · 2026-08-24
- AI 假记忆：为何模型越用越错，真记忆需遗忘 — PrajwalTomar_ · 2026-08-24