Surge AI's DAYJOB benchmark: best agents pass only ~24% of real work
echen · x · 2026-09-24
- Surge AI launched DAYJOB, a benchmark suite for professional knowledge work, debuting with DAYJOB: Healthcare and DAYJOB: Finance — 130 expert-level assignments total.
- Motivation: GDPval-style tasks come with detailed instructions, but real work is a vague "@bob is that ready yet" — agents must reconstruct the task from spreadsheets, docs, and Slack threads and decide which sources to trust.
- Results: the strongest model passes only 24.7% on Healthcare and 23.9% on Finance — frontier agents remain far from surviving a 9 to 5.
Related event: Surge AI Releases DAYJOB Benchmark; Top Agents Pass ~25%(3 posts)→
More from Research
- MIT's Tegmark coins 'formal-washing': Signal protocol proof is far from complete — tegmark · 2026-11-12
- Signal formal verification hits 80%, Tegmark says Wired overstated completion — tegmark · 2026-11-12
- Category-theoretic framework asks: when does a physical system actually compute? — neurovium · 2026-11-12
- Radical Numerics brings genome language model Omnii-1 to the edge for biodefense — exnx · 2026-11-12
- Against 'verifiable vs. fuzzy': the real axis is how expensive tasks are to grade — dfrsrchtwts · 2026-11-12
- Toby Ord flags 'all-things-considered' AGI forecast table is actually conditional on no pacing — tobyordoxford · 2026-11-12