Ofir Press: No good human-in-the-loop SWE benchmark exists; two futures for agents
OfirPress · x · 2026-11-06
Ofir Press argues there are no strong 'human-in-the-loop' SWE benchmarks today. If one emerges, tools like Claude Code/Codex might beat autonomous scaffolds like mini on it. He also sketches the opposite future: as humans supply less input across the SWE lifecycle, autonomous scaffolds like mini keep winning long-term.
More from coding & agent
- Executor v2 launches: Zapier/Retool-like automation fully drivable by code and AI agents — threepointone · 2026-11-06
- Y Combinator-backed Reflex raises $14M Series A, its Python framework hits 29k GitHub stars — ycombinator · 2026-11-06
- One diagram maps all the agent protocols: MCP, A2A, UCP, AP2, A2UI, AG-UI — rseroter · 2026-11-06
- Sutra: open-source terminal coding agent, lowest cost per solved task — cneuralnetwork · 2026-11-06
- From Greasemonkey to now: AI lets you mod any app just by describing the change — msfeldstein · 2026-11-06
- AI agent handling an office move spams movers with hundreds of questions until they refuse to reply — flngr · 2026-11-06