Enabling Creative Exploration for Vibe Design Agents
Yifan Zhang, Nghi D. Q. Bui, Georgios Evangelopoulos, Arnaud Benard
cs.AI
2026-09-14
Google samples scored UI themes then generates at fixed decoding. Coverage rises from 1.0 to 2.9; 300k+ online tasks cut negative feedback 31%, export lift still uncertain.
Vibe design agents such as Lovable, v0, Bolt, Figma Make, Claude Design, and Google Stitch already turn a brief into a rendered page and frontend code. Exploration is the remaining jam. Run the same request five times and the outputs collapse toward one conventional palette, type pairing, and composition. Users want a few coherent directions before they commit. The agent keeps returning the same face with a slight tweak.
Raising token-level temperature is a cheap knob, and a blunt one. Aesthetic choices and syntax-sensitive code share the same control. Color and radius move; markup and stylesheets jitter with them. A verbal "be creative" instruction still leaves the runtime without a distribution it can inspect or rebalance. The paper wants a separate control point: change the design direction, keep downstream decoding fixed.
The pipeline splits into three stages. The object between them is a structured design specification.
A proposal pre-pass, following Verbalized Sampling, asks the model for K=3 distinct, prompt-compatible directions. Each candidate carries structured attributes and a self-reported typicality score. The elicitation labels are Safe, Premium, and Experimental. Those labels are prompting vocabulary, not measured quality or risk.
Selection normalizes nonnegative scores into weights, then samples with a candidate-level temperature τ. At τ=1 the selector follows the elicited weights. Values below 1 concentrate on high-weight directions; values above 1 give lower-weight directions more mass. Zero-weight candidates stay unselected. Temperature can only reweight the proposed set. It cannot invent a direction the pre-pass never listed.
Generation feeds the selected specification and the original request into the existing path. Downstream model config and decoding stay fixed across selection conditions.
A theme specification binds seed color, light or dark mode, headline and body fonts, and corner roundness, and the generator is told to implement that bundle together. An asset specification is an image-generation prompt covering subject, composition, and style. The two interventions run separately in the main experiments, and jointly only in a meal-planning dashboard case study. Scores are treated as operational weights, not calibrated probabilities, because prompted numbers can diverge from next-token probabilities. Invalid scores trigger a re-prompt; if that still fails, the system falls back to the baseline path.
Gemini 3 Flash handles proposal, design-system generation, and code. Nano Banana 2 generates in-page images. Pairwise judging uses Gemini 3.1 Pro with UI-specific rubrics. Tested τ values are {0.5, 1, 1.5, 2, 5}. Product defaults are 1.5 for themes and 2.0 for assets, which are not the best settings in the offline sweep.
Offline evaluation covers 168 prompts: 83 short, open-ended items in a Standard UI Benchmark and 85 constrained items in a Complex UI Benchmark. Each temperature and each intervention has 1,255 paired comparisons.
On the standard theme study, observed option coverage across five repeats rises from 1.00 to 2.10–2.94. At τ≥1.5, none of the 83 prompts shows only one option. Screenshot embedding cosine similarity falls from a baseline of 0.6765 to 0.5438 at τ=2.0. Judge preference does not track that curve:
| τ | Options | Screenshot sim. | VS win / base win | W/L |
| 0.5 | 2.10 | 0.6560 | 33.6% / 41.3% | 0.81 |
| 1.0 | 2.59 | 0.6085 | 36.1% / 38.8% | 0.93 |
| 1.5 | 2.84 | 0.5844 | 34.1% / 40.1% | 0.85 |
| 2.0 | 2.87 | 0.5438 | 38.8% / 35.3% | 1.10 |
| 5.0 | 2.94 | 0.5437 | 35.1% / 38.8% | 0.90 |
τ=2.0 is the only theme setting that beats baseline on the judge, at a 1.10 win/loss ratio (38.8% vs 35.3%). τ=5.0 has nearly the same screenshot separation, and the preference flips. Compiled HTML DOM similarity drops from 0.8779 on baseline to 0.7935–0.8339 under the intervention: structure moves, the skeleton stays.
With theme sampling off and only image prompts sampled, whole-screen similarity barely moves (0.6616 vs 0.6765 at τ=1.0), yet the judge win/loss ratio reaches 1.32 (42.8% vs 32.3%). Raising asset temperature further helps neither measure. On the complex asset suite the best aggregate ratio is 1.19 at τ=1.5, so the standard-suite optimum does not transfer.
The online A/B sits on a commercial UI design assistant: 155,338 control tasks and 154,532 treatment tasks, about 506k generated screens. The share of tasks finishing within 60 seconds falls from 49.9% to 48.2% (relative -3.47%, 95% CI excludes zero). Task success moves from 97.95% to 97.80%. Code exports per screen go from 1.06% to 1.15% (+8.23%), but the interval [-13.03%, +29.48%] includes zero. Negative feedback events drop from 73 to 50 (-31.51%); the positive-to-negative ratio moves from about 3.8:1 to 5.7:1. Flagged corrections in an evaluated conversation subset rise from 38.8% to 41.6% (+7.08%). Fewer complaints, more steering, slightly worse latency: all three hold at once.
For people shipping vibe-coding or UI agents, the usable idea is an inspectable intermediate decision. Decoding temperature can stay put. Theme sampling moves the whole-page look; asset sampling moves local semantics that whole-screen similarity barely sees and an LLM judge still scores. Exploration systems need coverage, visual separation, judge preference, and live behavior reported apart.
This is incremental systems work. Verbalized Sampling already supplies the elicitation trick. The contribution is wiring a structured spec into theme and asset generation, then measuring it offline and in production. Product default τ values do not match the offline peaks, so the knob still has to be tuned per prompt suite.
The comparison is the full proposal-and-selection pipeline against baseline. It does not separate typicality weights from the act of proposing multiple candidates. Missing controls include unweighted elicitation, uniform selection, source-faithful VS, and a token-temperature change. Archived records also do not show whether identical candidate sets were reused across temperatures and repeats, so the τ results describe whole configurations.
Coverage counts how many of three options appear across five repeats and does not say whether option identity is a persistent specification or a tier label. The authors already warn that this is not a count of distinct rendered designs. LLM judges carry position, verbosity, and same-family bias, and the offline aggregates lack prompt-clustered uncertainty. Pairwise judging uses Gemini 3.1 Pro on Gemini 3 Flash outputs; same-family preference is not ruled out.
The online snapshot uses a PREPOST interval method and does not document treatment configuration, exposure dates, or randomization well enough to reconstruct. Voluntary feedback totals 351 and 335 ratings. Prompts, generations, evaluator records, and executable analysis are unreleased. There is no blinded professional design rating, and no direct execution or accessibility check that the downstream output actually honored the selected spec.