Rewriting an agent skill: A/B challenge runs show skills should only add what agents don't know
jmschreiber91 · x · 2026-10-21
jmschreiber91's team fully rewrote the tangermeme agent skill around one principle: a skill shouldn't tell the agent what it already knows. They validated it with challenge runs — agents completing tasks with and without the skill — measuring token usage and wall-clock time.
The rewritten skill leans on deterministic scripts agents can call, and focuses on non-obvious pitfalls and warnings. The methodology (skills carry only incremental info, trimmed by measured A/B runs) is directly applicable to anyone authoring Claude skills or agent prompt docs.
More from coding & agent
- UC Berkeley releases Actor-Gatherer framework decoupling reasoning from context management — adityagp · 2026-10-21
- Hypothesis Dreamer Agent: A Weekly AI Prompt for Generating Research Ideas — nlarusstone · 2026-10-21
- Dev Builds 'Sandman' Grok Bot to Manage Context Across a 33-Bot Fleet — alexcovo_eth · 2026-10-21
- Hallance launches: an AI aiming to do the work of an entire IT consultancy — TAbrodi · 2026-10-21
- SherpaLabs launches RL environments for finance, coding, quant; top models pass only ~6% on its finance benchmark — ycombinator · 2026-10-21
- The 20x cost trap: AI agents cost 10x traditional AI and 20x conventional software — DavidLinthicum · 2026-10-21