Scaling to 128 agents lifts pass rate from 19.3% to 28.8% on hardest ProgramBench tasks
OfirPress · x · 2026-09-29
The Agensh paper scales from 1 to 128 agents on the 5 hardest ProgramBench tasks, using GPT-5.6-sol (high reasoning) with Copilot as the single-agent harness. Mean 6-hour final pass rate rises from 19.31% to 28.78% — a 49% relative gain. Ofir Press highlights open questions: hierarchical vs flat organization, self-organized vs prescribed structures, and whether to pre-assign agent roles.
More from coding & agent
- Continual-ARC: lifelong ARC-AGI stream finds agent harness persistence rarely beats simple caching — unifyai · 2026-11-16
- WebArena-Pro: a 300-task multimodal benchmark across 20 self-hosted apps for web agents — McGill-NLP · 2026-11-16
- Filmworkbench v0.1.2: one-prompt semi-automated short films with Claude and ComfyUI — Eastern-Ad9515 · 2026-11-16
- Stop prompting LLMs for JSON: PowerShell module Jev routes with decision models — dfinke · 2026-11-16
- llama.cpp offloads CI caches and runners to Hugging Face to survive 35 PRs/day — ggerganov · 2026-11-16
- I killed my 30-minute weekly sync with a decision log my agents read before every run — Humaux · 2026-11-16