Surge AI launches DAYJOB: Tax benchmark — Opus caught the deadline 5/5 times, Astra never did
echen · x · 2026-10-28
Surge AI's Edwin Chen argues most benchmarks are too sanitized: they test well-defined problems, while figuring out what the problem is half the job. The new DAYJOB: Tax benchmark injects real-world messiness into professional tax scenarios. In one case, a good advisor should proactively warn that a home sale delayed past August 2027 would forfeit the $250K capital-gains exclusion. Opus flagged the deadline in 5/5 runs; Astra never did. DAYJOB previously covered Healthcare and Finance.
Related event: Surge AI Launches DAYJOB: Tax Benchmark for Real-World Agents(2 posts)→
More from Models
- Unreleased model reportedly solves ETH and Quasi-Riemann problems that current models can't — BlancheMinerva · 2026-10-28
- Uncensored LFM3-1.7B GGUF quants drop refusal rate from 61.7% to 0% — alexcovo_eth · 2026-10-28
- CRUX shadow eval: GPT-6 Astra and Claude Fable 5.1 research papers both rejected by experts — patpat_mit · 2026-10-28
- Ollama Launches Cloud Service with Crusoe to Run Frontier Open Models — ollama · 2026-10-28
- How a European AI lab post-training Chinese open models can earn credibility — alejandroll10 · 2026-10-28
- GLM 5.3 Flash NVFP4 on 4x RTX 6000 WS: V2 serving nearly doubles throughput per GPU — TheZachMueller · 2026-10-28