Open-Source Tool for Validating Generative UI
zook1337 · reddit · 2026-07-15
The author introduces an open-source project uivet for validating non-deterministic generative UI in CI.
Problem
Generative UI output varies each time; pixel diff regression isn't applicable, and LLM-only text evaluation doesn't render DOM.
How it works
For each scenario:
- Samples N generations
- Renders with headless Chromium
- Evaluates data consistency, accessibility (axe-core), layout, console errors
- Combines with a multimodal judge model for strict JSON output
- Uses score stddev for consistency and fails CI via exit code comparison with baseline
Design motivation
Real defects: generated cards silently omit warranty links. Human evaluator agreement (Cohen's kappa) is low, necessitating parallel deterministic checks and model judges.
Current ask
This is an MIT-licensed project with an offline demo. The author seeks critique on:
- Whether multimodal JSON scoring is stable enough to block builds
- Or if we should only trust deterministic consistency and integrity thresholds
More from coding & agent
- Coding agents are heading toward an AI-writes, AI-reviews, human-approves workflow — aftahi_ai · 2026-07-22
- oMLX 0.5.2 adds Mac menu-bar stats, low-bit decode kernels, and faster downloads — awnihannun · 2026-07-22
- GitHub review bot hits its PR limit and forces a 39-minute cooldown — DanielLockyer · 2026-07-22
- Max reasoning effort appears to be mobile-only in Codex Remote, not desktop — GabGarrett · 2026-07-22
- A Reddit demo argues online stores should expose carts and pricing through MCP — gelembjuk · 2026-07-22
- Open-source AI SDK provider routes Vercel apps through a local Codex subscription — lgrammel · 2026-07-22