AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design
Yaxin Luo, Haobin Jiang, Jialv Zou, Xu Huang, Wenhao Yan, Haodong Li, Zhengrong Yue, Jing Li, Xiaofu Chen, Xiaohan Zhao, Jiacheng Liu, Jiacheng Cui, Zhiqiang Shen, Xiaotong Li
cs.CV, cs.AI, cs.CL
2026-08-14
AutoDesign lets a coding agent recursively rewrite its own poster-generation harness, without touching model weights, lifting PosterBench scores by 12.4% and beating Claude Design by 7.45 points.
Turning multimodal sources like academic papers into polished outputs (posters, slides, webpages) is fundamentally a long, multi-step agentic process. The system around the model (the harness: prompts, tools, evaluation feedback) ought to get better at this over time, but most existing systems are static: they don't learn from past runs. AutoDesign asks whether a coding agent can recursively improve the harness that produces these artifacts, rather than repeatedly polishing one output at a time.
AutoDesign runs two nested loops. The inner loop is the design harness itself: a designer module generates or revises an artifact, a critic (a rule-based validator plus a VLM visual critic) gives feedback, and this repeats for up to 12 attempts or until the candidate passes validation. The outer loop, the meta-harness, treats the design harness as the thing being optimized: it runs the current harness across a training set of tasks, collects trajectories and scores, then has a coding agent (acting first as planner, then as code editor) analyze recurring failures and propose one bounded change to one of five harness components (context and memory, tools and specs, execution runtime, orchestration, or evaluation and feedback).
The key constraint is an acceptance gate: a candidate change is only adopted if it improves the score on the training set and does not regress the score on a held-out development set; otherwise it's discarded and the harness stays as-is. Restricting each iteration to a single component means any score change can be attributed to one specific edit rather than several tangled together. Humans can optionally redirect the search with natural-language guidance, or fix a systematic bias in the evaluator when the VLM critic misses something, but they don't edit the harness code directly.
The paper validates this on paper-to-poster generation (PosterBench: 100 papers across five disciplines, scored on seven dimensions: faithfulness, coverage, density, visual evidence, layout, readability, aesthetics).
| System | PosterBench Score |
| AutoDesign (Claude Code + Claude 4.8) | 78.32 |
| AutoDesign (Codex + GPT-5.5) | 77.97 |
| Claude Design (matched setup) | 70.87 |
| OpenDesign (matched setup) | 69.45 |
Under the exact same Claude Code + Claude 4.8 configuration, AutoDesign beats the commercial system Claude Design by 7.45 points. Attaching the learned DesignHarness to seven different model/coding-agent combinations raises the average score from 54.99 to 67.39, gains ranging 5.0 to 19.6 points, with the largest lift (19.6 points) on the comparatively weaker DeepSeek V4 Pro. Eleven blind reviewers made 936 pairwise judgments; AutoDesign's Bradley-Terry win probability came out to 64.0%, and when the two posters differed by 20+ PosterBench points, human preference agreed with the score 74.4% of the time. In a fully autonomous run, it executed 253 tool calls and 11 editing turns in 40 minutes for under $3, reaching conference-poster quality.
The core move here, shifting the optimization target from is this one output good to is the system that produces outputs good, generalizes past posters to any workflow that repeatedly produces content with an agent. The concrete recipe is worth stealing: an acceptance gate (train-set gain, no dev-set regression) plus one-component-at-a-time updates solves the two failure modes that usually sink self-editing agents — silent regressions on other tasks, and unattributable changes when several edits land at once. And because the thing being optimized is the harness, not model weights, the same learned DesignHarness transfers cleanly across seven different underlying models, which is evidence it captured reusable process knowledge rather than a model-specific patch.
The authors are upfront that PosterBench only rigorously evaluates posters; slides, webpages, and videos, which the meta-harness framework nominally supports, only get an illustrative 4x4 demo rather than the same scored evaluation. The acceptance gate only guarantees no regression relative to a training and development set drawn from the same 100-paper distribution, so generalization to very different paper styles (non-English papers, ultra-short workshop papers) is untested. The human preference confidence interval, 55.2% to 77.8%, is wide enough that 11 reviewers and 936 judgments aren't quite enough to pin down the exact margin of preference. And the meta-harness optimizer itself is a strong off-the-shelf coding agent (Claude Code); whether the self-improvement loop still works with a weaker coding agent as the optimizer is never tested.