Google's WikiSkill: a 9B model beats a 27B one, but one bad skill craters Gemini to 18.1%
montymike · reddit · 2026-08-31
Google Research's new WikiSkill preprint proposes an approach for long-running agents: instead of relying on model weights or a growing transcript, consolidate execution history into a maintained knowledge base and compile it into reusable skill files.
- Three layers: immutable raw traces; a "wiki" consolidating successful strategies, recurring failures and past changes; a proposer distilling lessons into concise skills, with a validation gate keeping only updates that improve immediate score
- Headline: across five agent benchmarks, Qwen-3.5-9B with evolved skills averaged 47.4% vs 39.4% for Qwen-3.6-27B without; the 27B hit 63.3% (+23.9pp) with its own skills — procedural memory and model capacity are complementary, not substitutes
- Mixed transfer: on ALFWorld the 9B scored 70.2% with the 27B's skill vs 63.4% with its own; but a spreadsheet skill from the weakest Qwen dropped Gemini-3.5-Flash from 50.5% to 18.1% — brittle workarounds from weak systems become harmful instructions for stronger models
Limits: bounded benchmarks, skills placed directly in prompt rather than retrieved from a library, no wiki pruning. Paper: arxiv.org/abs/2608.27454
More from coding & agent
- Harness, a command center for coding agents, hits #2 on Product Hunt — dee_hw · 2026-10-16
- nanobot: a 4,000-line rewrite of openclaw, 99% smaller — mdancho84 · 2026-10-16
- Open-source Free WebAccess routes agent web search across vendors for $0 with auto-fallback — yangyi · 2026-10-16
- State of Agentic Coding podcast hosts Geoffrey Huntley on code mode and self-modifying harnesses — mitsuhiko · 2026-10-16
- The Road to Self-Improving Software: Building a Closed-Loop Agent Factory — _AustinCalvert_ · 2026-10-16
- Screenshots show "talk-to-build" AI app tool with one-click open-source publishing to GitHub — yangyi · 2026-10-16