Gemini 3.7 Flash tops DeepORG benchmark with 97.3/100 score, zero unsafe actions
dosco · x · 2026-08-31
GraphJin released the DeepORG benchmark for organizational agents, testing 113 real database tasks.
Key Results:
- Gemini 3.7 Flash: Achieved a 97.3/100 full pass rate with 96.5% policy adherence. Crucially, it recorded zero unsafe effects (no unintended writes or side effects) across 339 attempts.
- Cost & Latency: Priced at $0.092 per attempt with a P50 latency of 13.8s.
- Scope: The benchmark evaluates not just correct answers, but also governed methods and safety compliance.
DeepORG aims to verify an agent's governance and safety when connected to live production databases.
More from coding & agent
- OpenClaw 2.0 Released: Rebuilds Installation and Browser Experience — heyneighbor · 2026-08-31
- MCP Server Lets LLM Agents Play Civilization VI Natively — jia_seed · 2026-08-31
- Hermes Agent running GLM-5.2 via OpenRouter on a 2GB VPS — granawkins · 2026-08-31
- Training custom agents with TRL + OpenEnv workflow — ariG23498 · 2026-08-31
- My own scrubber was bypassed by the very next line — leak survived 13 releases — Thirumalaiboobathi · 2026-08-31
- Plan ordering decides success in 9-week codegen experiment — Ok_Astronomer_526 · 2026-08-31