EnterpriseShift: a benchmark testing agents on role-based enterprise work across 13 jobs
Shahules786 · x · 2026-11-04
Vibrant Labs released EnterpriseShift, a benchmark for role-based enterprise work, with a detailed design blog.
- Critique of existing benchmarks: task-centric environments preselect context and narrow dependencies (doing part of the agent's reasoning), and none define explicit job responsibilities or runtime role-based access control.
- World building: 13 defined jobs, 300 policy documents (1.5M tokens), 6 simulated systems (ServiceNow, Entra ID, Entra ID Governance, Teams, SharePoint, StatusGator), records grounded in real ITSM activity logs including unfinished work and historical irregularities.
- Authority constraints: a Change Manager can authorize a change but cannot implement it—agents must coordinate with simulated colleagues.
- Tasks: agents own an entire shift, discovering what needs attention in mixed queues and applying policy; verification checks outcomes, action order, authorship, and handoffs; completion may mean resolving, escalating, or waiting and resuming.
A comparison table shows Corecraft, HANDBOOK.md, EnterpriseOps-Gym, τ-Banking, and AutomationBench all lack job definitions, runtime RBAC, and asynchronous coordination.
Related event: Vibrant Labs Launches EnterpriseShift Benchmark for Enterprise Agents(4 posts)→
More from coding & agent
- Flea 0.4.0 file manager for Omarchy rebuilt in GPUI with TUI and plugins — mark_k · 2026-11-04
- GhostDraw: a hobbyist turns hand-drawn sketches into app diagrams with tldraw and AI — Dhananjay_Tech · 2026-11-04
- CoworkOS Debuts: Three-Layer Agent Coworker Framework Lifts Qwen3.6 by 9.76 Points — KyeGomezB · 2026-11-04
- DOLORES: Training-Free Meta-Scaffold Lifts 8B Models Past 32B Baselines — StellaLisy · 2026-11-04
- Berkeley-affiliated StarTrail open-sources a deep research benchmark and paper-search toolkit — CShorten30 · 2026-11-04
- AI job-search framework built on Claude Code hits 69.3k GitHub stars — adnan_hashmi · 2026-11-04