Agent S4: a 27B model jumps 5.6% to 21.3% on OSWorld without any weight changes
krishnan · x · 2026-10-26
Simular's Agent S4 paper shows a 27B open model going from 5.6% to 21.3% on OSWorld V2 without touching its weights—only the tooling around it changed.
Mechanism: a meta-agent reads the task agent's trajectories, then writes and revises the interfaces it works through—bulk spreadsheet edits instead of cell-by-cell clicks, page text pulled from the logged-in browser tab instead of screenshots. It scores above GPT-5.5 (13.0%) and Opus 4.8 (20.6%) at $1.10 per task versus $25.50 and $72.40.
Caveat: authors' own numbers on 108 tasks, not independently reproduced. The enterprise read: the tool layer around agents is now software agents write themselves—a change-control problem: who reviews an agent-built interface before it touches the ERP?
Related event: Simular's Agent S4 Beats GPT-5.5 by Evolving Interfaces, Not Weights(2 posts)→
More from coding & agent
- SF 'Code Engineering' event picks up where 1,000-person harness engineering meetup left off — LangChain · 2026-10-27
- Prompt turns Claude Code into an iPhone backup forensics tool exposing 30-day 'deleted' photo copies — alex_verem · 2026-10-27
- Prompt turns Claude Code into a forensics tool finding deleted iPhone photos in backups — alex_verem · 2026-10-27
- Phones won't be mobile IDEs but control planes for agent fleets, argues Yuchen Jin — Yuchenj_UW · 2026-10-27
- FrontierCS 2 launches on Kaggle: 48 tasks make agents beat published CS papers — MeganRisdal · 2026-10-27
- Stop remaking old games with AI: a call for more ambitious agentic software — rschu · 2026-10-27