Cognition launches FrontierCode 2.0, a coding benchmark testing judgment beyond the spec
DevinAI · x · 2026-10-28
Cognition's Devin team, working with its research team, released FrontierCode 2.0, the successor to a benchmark built around the question "would you actually merge this PR?"
- Version 2.0 features more diverse tasks, larger scopes, and higher mergeability standards — asking not just whether you would merge a PR, but whether you should
- Key feature: deliberately ambiguous, underspecified prompts that leave engineering decisions open, giving models room to demonstrate judgment
- Announced via DevinAI's account citing Cognition's official launch post
More from Research
- SG-Splat: Pose-Free Feed-Forward 3DGS Decouples Geometry and Attributes for Stability — zhenjun_zhao · 2026-10-28
- CVPR 2026 Highlight: Pose Graph Optimization Fixes ACE-Zero Drift, Matches COLMAP — zhenjun_zhao · 2026-10-28
- VGGT Knows More About Its Pose Failures Than It Reports: Linear Probe Hits 93.5% AUC — zhenjun_zhao · 2026-10-28
- TLP: Single-Parameter Point Cloud Registration with Truncated Lp Functions — zhenjun_zhao · 2026-10-28
- Dumb chunk-and-rerank baseline beats top agentic memory systems in 6 of 7 benchmarks — JeremyCMorgan · 2026-10-28
- Microsoft's MANDATE: 23 failure categories from 163k Reddit posts on AI agents — dair_ai · 2026-10-28