Probability-scored filtering with a small model beats LLM summarization for RAG and context compaction
marlene_zw · x · 2026-09-23
The author shares a hands-on technique for context compaction in the Copilot SDK: instead of one-shot summarization with Opus or GPT, a small model (Jev) scores each passage with probabilities and keeps/drops by thresholds — P(relevant) < 0.45 → drop, P(has answer) > 0.55 → keep. They're now extending it to 4-level scoring: probability still needed for the goal, superseded by later activity, and needed again. Cheaper and faster, though final output still needs evaluation.
More from coding & agent
- Claude Opus system prompts keep growing: Opus 5.5 hits 4,108 words, mostly refusal guidance — rajistics · 2026-09-23
- WorkSwarm pitches real agent teamwork: persistent sessions, model routing and RSI — HeyToha · 2026-09-23
- Open-Source WorkSwarm Builds AI Agent Teams With Persistent Sessions and Recursive Self-Improvement — HeyToha · 2026-09-23
- Yacine asks: is anyone building an RL environment for code smells? — yacineMTB · 2026-09-23
- Qwen3.8-Flash Free in Qoder: Zero Credits for Individual Users, 14-Day Pro Trial With 300 Credits — mhdfaran · 2026-09-23
- Qoder walkthrough: start with one outcome, skip juggling five tools — mhdfaran · 2026-09-23