Gemini 3.7 Flash tops DeepORG benchmark with 97.3/100 score, zero unsafe actions
dosco · x · 2026-08-31
GraphJin released the DeepORG benchmark for organizational agents, testing 113 real database tasks.
Key Results:
- Gemini 3.7 Flash: Achieved a 97.3/100 full pass rate with 96.5% policy adherence. Crucially, it recorded zero unsafe effects (no unintended writes or side effects) across 339 attempts.
- Cost & Latency: Priced at $0.092 per attempt with a P50 latency of 13.8s.
- Scope: The benchmark evaluates not just correct answers, but also governed methods and safety compliance.
DeepORG aims to verify an agent's governance and safety when connected to live production databases.
More from coding & agent
- Fintech HR built a sourcing agent on 60,000 past applicants — 30 qualified candidates in a month — G0NZ0BOY · 2026-10-16
- Orchestrating a 100-image portrait of humanity for aliens with ChatGPT, Argon and p5.brush — dx8xb · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Builder's one rule for trusted desktop agents: read freely, but sending, deleting, buying always ask first — No_Fan5310 · 2026-10-16
- GPT-built unifi-simulator makes MikroTik switches appear inside UniFi Network — Pytlicek · 2026-10-16
- Stanford drops 45-min SpaceX lecture on how Grok Bot's always-on agent harness works — Hesamation · 2026-10-16