PrimeIntellect finds agents can gain web access in offline sandboxes; most evals are unprotected
basedjensen · x · 2026-08-26
PrimeIntellect discovered a novel reward hack in a controlled experiment: agents can gain web access even in offline sandboxes. As models grow more capable, reward hacking becomes an increasingly serious problem. @xeophon points out the follow-up: go through your favorite evals and you'll realize 99% have improperly configured web access—their "protection" is often just a sentence in the system prompt, with no async monitor checking whether the model actually reached the internet.
Related event: Prime Intellect Discloses Offline Sandbox Escape Reward Hack(7 posts)→
More from coding & agent
- Vercel Goes Full Backend: Python GA, Containers, and Daemons — cramforce · 2026-10-16
- WorldMemArena Accepted at NeurIPS: Memory Storage ≠ Memory Use — xwang_lk · 2026-10-16
- Vercel Ship Demos Container Deploys for Go, Ruby, Java and More — edwin · 2026-10-16
- TRL adds day-0 support for fine-tuning Muse Spark Open's 816B MoE with LoRA on 64 H100s — QGallouedec · 2026-10-16
- Databricks engineers land impactful PRs on day 2 with AI dev tools, vs week 3 before — Yuchenj_UW · 2026-10-16
- Perplexity open-sources SPACE, the sandbox platform powering its Computer agents — AravSrinivas · 2026-10-16