Surge AI launches DAYJOB: Law with 71 lawyer-written tasks; Gemini 4 spots a $20M issue then ignores it

echen · x · 2026-11-19

Surge AI released DAYJOB: Law, a benchmark of 71 assignments written by practicing lawyers, spanning corporate transactions, litigation, criminal defense, employment disputes, IP, and estate matters. Agents face realistic client requests, messy case files, and little hand-holding, and must investigate facts, apply the law, and produce lawyer-usable work.

In one case study, Gemini 4 Carbon was asked to review a startup's records for a $20M investment and calculate the ownership stake. It uncovered a critical discrepancy — 350,000 shares held by a former executive missing from the ownership spreadsheet but still on the official stock ledger, with a board-approved buyback but no proof it happened — then ignored its own finding, computing a precise $3.3526 per share assuming the disputed shares were cancelled. That's the benchmark's core question: can agents move beyond finding facts to exercising professional judgment when millions are on the line? echen notes Harvey's legal research team collaborated on the benchmark.

Related event: Surge AI Launches DAYJOB: Law Benchmark with 71 Real Legal Tasks(2 posts)→

Original post →

More from Models

Models channel →