Questioning if ExploitGym's scorer can detect poisoned agents
moyix · x · 2026-08-28
The author questions the published scorer in the ExploitGym repository, which uses an LLM judge to verify causality, asking whether it would indeed disqualify agents that were effectively “poisoned”.
Related event: ExploitGym's LLM Judge Fails to Catch Poisoned Agents(2 posts)→
More from coding & agent
- raylib 6.2 ships with 300+ improvements and 100 new contributors, 20 new API functions — jsuarez · 2026-10-16
- "Vibecoding" is as misleading a label as "AI" and "SI," dev complains — eschadiol · 2026-10-16
- Dev runs Pstack as a Grok bot to audit his own pipeline — it surfaces a pile of hidden bugs — alexcovo_eth · 2026-10-16
- Doc2Harness: New benchmark finds best coding agent passes only 53% across 329 cases — _akhaliq · 2026-10-16
- Perplexity open-sources SPACE, the Firecracker microVM sandbox behind Perplexity Computer — perplexity_ai · 2026-10-16
- Blackstone goes all-in on Factory for autonomous software development — matanSF · 2026-10-16