Test if knowledge is in the weights: have the model write a skill, then delete it
dexhorthy · x · 2026-10-26
dexhorthy proposes a practical alternative for checking whether knowledge is already internalized in a model's weights:
- Ask the model to write a detailed skill for a task with very light steering — essentially the natural-language message you'd normally use to trigger the skill.
- Read the created skill contents (drawn primarily from the weights plus any light harness-supplied context, e.g. about your codebase).
- If the contents look correct, delete the skill — it's in the weights.
A hands-on answer to the question of how to tell what a model already knows without external scaffolding.
Related event: Debate: Does Self-Written Skill Deletion Test Prove Knowledge in Weights?(2 posts)→
More from coding & agent
- Sharing memory across Claude, Codex and Cursor: one git repo of Markdown as agent memory — jordimr · 2026-10-26
- Suno-MCP: open-source MCP server lets AI assistants generate music via Suno API — modelcontextprotocol · 2026-10-26
- LangChain explains how it builds targeted evals for Deep Agents — sharpeye_wnl · 2026-10-26
- Pointer: open-source MCP server lets AI agents drive your real Chrome tabs — FinancialManager7861 · 2026-10-26
- Open-source Chowbot MCP server lets you order food via ChatGPT or Claude — saheedniyi_02 · 2026-10-26
- What if all human-written code goes in .gitignore? Redditors debate dev's future role — michal-rd · 2026-10-26