A Decade in a Day: Synthesizing Agent-Native Interfaces from Human-Oriented Software Evolution
Alan Wu, Wei Tao, Yiwen Guo
cs.SE, cs.AI
2026-10-12
Version history as a natural experiment shows newer interfaces consistently help agents; ANIS, built on those findings, reaches 74.5% success with about 73% fewer tokens.
LLM agents are becoming a new class of users of desktop applications, but Blender, QGIS, and FreeCAD expose functionality through interfaces built for human eyes and for developers. GUIs assume a user who can see controls and read visual feedback; SDKs assume someone who will search docs, write code, and debug low-level calls. An agent gets neither. It either parses pixels or gropes through sparsely documented SDKs, composing long chains of low-level operations. The interface mediates every action an agent takes, and a capable model handed an interface that hides the needed operations cannot finish the task.
Studying what makes an interface agent-friendly needs realistic, controlled variation, and hand-crafted variants would just encode the researchers' assumptions. The authors treat version history as a ready-made natural experiment: core functionality stays stable across releases while the interface keeps being reshaped under real usability pressure. Fix the task, agent, and verifier, vary only the release, and the performance shift traces to documented interface changes.
Three stages. First, DeskIF-Bench: 255 tasks across six applications in five domains (Writer, Impress, Inkscape, Blender, QGIS, FreeCAD), each with a deterministic script verifier that inspects the saved artifact rather than the screen, so the same task runs unchanged against different releases. 102 tasks are filtered from OSWorld and OpenComputer; 153 are synthesized by an authoring agent that gets privileges the solving agent lacks, such as application skills and libraries like trimesh, keeping task difficulty above the solvers' comfort zone.
Second, the cross-version experiment: one scaffold (OpenCode), two backbones (Claude Haiku 4.5, DeepSeek V4 Flash), four historical releases per application, scored with pass@3. Trajectory analysis distills three interface properties:
Third, ANIS operationalizes the properties. At build time it mines capabilities from the running application (registered operations, introspected objects and writable properties, menu items), test-executes each candidate, and keeps only invocations whose effects survive a save-and-reopen as atomic commands, pairing every writable property with a read command and grouping commands by artifact object so no external docs are needed. Granularity cannot be settled at build time, since worthwhile goals only show up in use: at runtime ANIS counts recurring atomic subsequences and promotes one to a goal-level command only after it recurs in at least three requests, maps to a shared subgoal, and replays cleanly on a fresh artifact copy. Interface generation takes 17.1 to 36.4 hours per application.
| Config | Success | Tokens/task |
| Native SDK | 68.5% | 84.4K |
| CLI-Anything | 17.4% | 67.0K |
| ANIS atomic | 72.3% | 22.6K (down 73.2%) |
| ANIS + goal-level | 74.5% | 21.2K (down 74.9%) |
DeepSeek V4 Flash backbone, 368 tasks, eight applications; the added Calc and GIMP were held out from all development. With Haiku the ordering is unchanged: atomic commands add 3.5 points over the SDK, goal-level commands reach 67.4%.
The version study: the newest release beats the oldest in all 12 model-application series. QGIS gains the most, DeepSeek 41% to 69%, Haiku 33% to 65%. On one graduated GDP map task, QGIS 3.10 forces the agent to derive eleven classes and assemble the renderer call by call: 33 turns, 662k tokens, failure. QGIS 3.44 ships QgsClassificationFixedInterval: 11 turns, 36k tokens, pass.
Two more findings. Banning observation commands while keeping editing commands drops success from 72.3% to 60.3%, so observability carries 12 points. And 98.7% of CLI-Anything's failures trace to commands its interface simply omits; ANIS exposes over 20x as many commands. CLI-Anything loses on coverage, not on the command-line form.
The headline contribution is measurement, not the synthesizer: interface design gets isolated as an experimental variable for the first time. Agents spend 24.1% to 36.2% of input tokens on SDK discovery, and the three properties work as a checklist for anyone building tool APIs, MCP servers, or agent sandboxes. ANIS is also practical: a verified command layer generated for any installed application, with roughly 73% fewer tokens, a direct cost line for token-billed deployments, and generation takes about a day, which is what the title's "a decade in a day" means. The version finding cuts both ways: a decade of human-oriented evolution happened to help agents, but the observability gap was never closed because no human user demanded it. Waiting for evolution does not work.
Author-acknowledged: ANIS with goal-level commands sits below the native SDK on FreeCAD (77.4% vs 96.8%) and Blender (51.4% vs 54.3%), because operations there depend on implicit object relationships (linked sketches, scene selections, edit modes) that automatic validation rejects; 28.6% of ANIS's FreeCAD failures lack a required command. Only 10 goal-level commands were synthesized, so their gain is modest (72.3% to 74.5%). Publishing goal-level sequences is left as future work.
Reading closely: the atomic layer leads the SDK by only 3.8 points on success, and the main win is tokens; the two should not be conflated. Generation relies on an LLM coding agent (both generated interfaces were built with Claude Opus 4.8) with no systematic ablation of that step. pass@3 overstates single-shot success. And cross-version gains cannot be attributed purely to interface: newer releases also add functionality and fix bugs; feasibility screening mitigates the confound but does not remove it.