Developer pushes back: recent context helps, so the "delete the skill" test isn't reliable
lucasmeijer · x · 2026-10-26
lucasmeijer challenges dexhorthy's "have the model write a skill, then delete it" test: having the steps in recent context helps even if the weights can produce those steps when prompted, so the test isn't reliable — akin to using /grill-me and blindly accepting all answers.
He admits he doesn't know what a better test would be, calling that an unsatisfying part of AI-assisted coding.
Related event: Debate: Does Self-Written Skill Deletion Test Prove Knowledge in Weights?(2 posts)→
More from coding & agent
- User Claims Codex Autonomously Reverse-Engineered Serum2 and Stripped Its License — Kyrannio · 2026-10-26
- colab-mcp lets AI agents control Google Colab notebooks directly — helloiamleonie · 2026-10-26
- Postiz, an open-source AI-agent social media manager, hits 37.9k GitHub stars — abhishek__AI · 2026-10-26
- Ben Thompson's 'The End of the Internet': when creation goes free, winners go small — tushaarmehtaa · 2026-10-26
- Six Frameworks for Grading Personal AI Agents, Compared Side by Side — sujingshen · 2026-10-26
- Killed our Vector DB for SQLite BM25 and wrote a 3.6ns Rust sentinel: lessons from multi-agent loops — OpenComplaint524 · 2026-10-26