Dev routes tool calls through a 421M model at 12ms to catch risky agent actions
looselyhuman · reddit · 2026-11-21
A developer building Pi's Auto mode uses a two-step inference check on tool calls: step 2 escalates to Qwen 3.5 4B (150ms per call, only for escalations), while step 1 needs a fast small model to flag risky calls (deletions, credential access, outbound transfers). Currently evaluating Laya 421M: given tool name + arguments, it flags effects and runs Q80 on a 3080 at 12ms per call. On a small playback set it caught 24/24 watched calls but false-flagged 4/6 benign reads, including a literal search for rm -rf. The author is soliciting comparable models.
More from coding & agent
- mnemos: a self-curating agent memory plugin for Hermes that fades unused memories — RileyRalmuto · 2026-11-21
- Study: evolutionary LLM search found valid solutions but picked invalid programs in 2 of 3 runs — Unusual-Spot-5846 · 2026-11-21
- Screen-share + voice via MCP into Claude: a faster UI iteration workflow — Different_Scholar_74 · 2026-11-21
- Tencent & KCL release GroupAssistBench: group-chat agents recall memories but fail to use them — 量子位 · 2026-11-21
- Open-source E2E-encrypted MCP server lets your phone message a local AI agent — uriwa · 2026-11-21
- Ex-GitHub leader's Maven course "Elite AI Assisted Coding" on sale through Sunday — intellectronica · 2026-11-21