CMU's Singularity-Bench: Opus 5.5 Scores 1.25, Beating Human Baselines
rohanpaul_ai · x · 2026-11-14
A Carnegie Mellon paper introduces Singularity-Bench, six end-to-end agent tasks from pre-training to building serving engines and cluster orchestration, scored against production tools on identical GPUs. Opus 5.5 + Claude Code hit a Singularity-Index of 1.25 (human baseline = 1), matching or beating humans on all six tasks; the other 10 agents averaged 0.58. Its serving engine earned $11.52/GPU-hour vs SGLang's $8.18. The paper also red-teamed 9/10 OWASP agentic categories and released Breakwater to block reward-hijacking attacks.
Related event: CMU Releases Singularity-Bench for Full-Stack AI Agents(2 posts)→
More from coding & agent
- Qwen 3.5 4B scores 10/10 running a coding agent fully offline on a 16GB MacBook Air — Ok-Brush8377 · 2026-11-14
- Ditching $500/Month SaaS Stack: One MCP Lets Claude Code Run Outbound for 91 Cents — aryanXmahajan · 2026-11-14
- FastMCP 4.2 Ships Skills Over MCP, Pairing Tools With Usage Guidance — jlowin123 · 2026-11-14
- Free guide: build a one-person agency powered by a team of AI agents — EXM7777 · 2026-11-14
- Dev reverse-engineers Waymo's private API into a Cloudflare Worker MCP for Claude — MicahBerkley · 2026-11-14
- Dev's fix for chaotic agent work: a feed-shaped todo list with action buttons — _rchaves_ · 2026-11-14