Agentic Artifact Creation: Systems, Evaluation, Principles, and Opportunities
Tianfu Wang, Zhezheng Hao, Xilin Xia, Lixin Liu, Mengkang Hu, Hongzhang Liu, Xi Chen, Ziyan Liu, Xiankun Lin, Weijia Zhang, Nicholas Jing Yuan, Hui Xiong
cs.MM
2026-08-28
This survey of 259 works defines agentic artifact creation as stateful construction redirected by mid-run observations. Difficulty tracks coupled decisions and repairable failures.
Direct generation is enough when a result is cheap to inspect and cheap to regenerate. Posters, repository patches, and playable scenes are not that class of deliverable. Change the content and layout, dependencies, or downstream constraints move with it. Checks often arrive late, and by then the system no longer knows which unit to touch.
Most adjacent surveys cut the field by model family, agent architecture, or a single modality. This one takes the delivered artifact as the unit of analysis and asks whether intermediate observations can redirect later work. If they cannot, the pipeline is still one-way, and a final inspection can only regenerate the whole object.
Agentic Artifact Creation is defined at the level of one construction episode. Three conditions must hold together: an AI system materially constructs or revises the deliverable; artifact or process state is carried across decisions; and at least one intermediate observation changes a later action, revision target, branch, or stopping choice. Assistants that only consume artifacts, workflows whose observations never touch control, and one-shot generators are out. Mixed-initiative systems stay in when those three conditions hold.
The functional loop has three roles, repeated until accept or stop:
The search covered arXiv, Google Scholar, Semantic Scholar, ACM DL, and IEEE Xplore from January 2023 through 20 August 2026, plus citation tracing and venue audits. After screening, 230 systems and 29 construction benchmarks remain, 259 records in total. Artifacts are grouped into six families: textual, 2D visual, audio, video, spatial, and behavioral (executable, state-dependent responses). Application settings (creative production, brand, education, professional work, scientific research, engineering design) are a separate axis.
Across families, construction difficulty tracks how tightly decisions are coupled and whether failures become visible while they remain repairable. Learned judges add little independent evidence when they share the generator's preferences or blind spots.
Evaluation is split into three non-interchangeable targets: the delivered artifact, the construction trajectory, and the construction system. On trajectories, Mr. Dre reports 16-27% regression in previously covered content or citation quality during later revisions. System properties are thinner: under the authors' definitions, only three protocols evaluate a property beyond task capability, each covering one property.
A public Design Arena snapshot from 19 August 2026 is used to show that an overall rank hides task gaps.
| Metric | Result | Contrast |
| Corpus | 259 | 230 systems + 29 benchmarks |
| Mr. Dre revision regression | 16-27% | previously covered content or citation quality |
| Protocols for system properties | 3 | one property each, beyond task capability |
| Kimi K3, Models Arena | top percentile on all five tasks | Claude Fable 5, GPT-5.6 Sol, GLM 5.2 cluster behind |
| Kimi K3, native Android | 54.3rd percentile | Claude Fable 5 at the top |
| GPT-5.6 Sol, Agents Arena | 87.5th on slides / 34.2nd on mobile apps | same snapshot, different tasks |
Benchmark maturity is uneven. Dedicated benches are thickest for 2D visual and behavioral artifacts; audio, video, and spatial rows often reuse protocols from system papers. Spoken audio has no dedicated construction system in the strict corpus.
Four principles follow: externalize commitments onto addressable state; set control boundaries where dependencies or consequences change; make feedback map to a feasible local repair; after a change, invalidate stale evidence and revalidate affected state. The fourth is flagged as comparatively sparse.
Anyone building an agent that writes posters, patches repos, or assembles scenes can use this as a filter: where is state stored, how is a failure localized, how fine is the repair, and does evidence go stale. Direct generation is not declared obsolete. When failures are cheap and there is little accepted state to keep, the extra control can cost more than regenerating.
Three practical takeaways. An LM judge is not independent acceptance. Splitting work across agents does not by itself make the artifact repairable. Trajectory regression should be reported on its own, not hidden inside a final-artifact score. A coded paper list is released with the survey.
This is a boundary-and-catalog survey. Its value is the comparison axes, not a new score on one family.
Query exports, per-source yields, and pre-reconciliation labels were not retained, so inclusion is hard to reproduce at the same cut. 2026 is counted only through 20 August. The fourth principle (selective revalidation) is explicitly thin. Table 4 is a representative set, not a frequency estimate.
The Design Arena numbers are an external leaderboard snapshot, not a controlled experiment by the authors, and ranks will move. The definition does the screening: systems that never let observations redirect later work never enter the 230, which are then used to support the definition. That loop is real. The six-family difficulty lens is qualitative; the paper says it is not a causal ranking.