safeagents live RL runs: decision models cut false blocks toward reward-hacking-free training
yb2698 · x · 2026-11-10
The safeagents project (huggingface/safeagents) is running final live RL training to eliminate reward hacking via decision models, with v3 focused on analyzing false positives from detector models.
- Leaderboard (balanced accuracy, 661 calls): Cloudflare clef-flash 0.893, clef-omni 0.877, vLLM d3-mini 0.856, LiquidAI d1-3B 0.836 with only a 0.8% false-block rate.
- Final runs: LFM2.5-24B-A2B (200 steps), LFM3-1.7B (400), TwIL-8B (100), on SmolDataEnvs and 230 traj2envs tasks; one LoRA adapter per model×harness, three trainers sharing one vLLM.
- Mechanism: a proxy inside the trainer inspects every tool call — undeclared tools, unknown hosts, or high-severity atif-scan matches get blocked (rollout scores 0); low-severity hits are logged and run normally.
- Method: open models train unmodified inside real harnesses (pi, opencode, hermes) to test whether RL on verifiable tasks improves performance under that same harness, tracked by held-out pass@1 and tool calls per correct answer.
More from coding & agent
- YC F26's 0.security Launches Autonomous Security Team That Ships Fixes, Claims 30+ Linux 0-Days — ycombinator · 2026-11-11
- DSPy 3.5 released: framework is now 85% smaller and production-ready — dbreunig · 2026-11-11
- A Swarm of 10 DeepSeek Agents With an Intercom Turns Out Extremely Effective at Finding Exploits — andersonbcdefg · 2026-11-11
- DIY AI neon sign: LILYGO display + Hermes Agent + LFM3-Audio voice terminal — alexcovo_eth · 2026-11-11
- Vercel Security Review flags 57,897 vulnerabilities in first week, agent auto-patches 14,665 — cramforce · 2026-11-11
- RLVR lets small models close the coding gap; Python's dynamic typing hurts pretraining — mdancho84 · 2026-11-11