Open-Source Tool for Validating Generative UI
zook1337 · reddit · 2026-07-15
The author introduces an open-source project uivet for validating non-deterministic generative UI in CI.
Problem
Generative UI output varies each time; pixel diff regression isn't applicable, and LLM-only text evaluation doesn't render DOM.
How it works
For each scenario:
- Samples N generations
- Renders with headless Chromium
- Evaluates data consistency, accessibility (axe-core), layout, console errors
- Combines with a multimodal judge model for strict JSON output
- Uses score stddev for consistency and fails CI via exit code comparison with baseline
Design motivation
Real defects: generated cards silently omit warranty links. Human evaluator agreement (Cohen's kappa) is low, necessitating parallel deterministic checks and model judges.
Current ask
This is an MIT-licensed project with an offline demo. The author seeks critique on:
- Whether multimodal JSON scoring is stable enough to block builds
- Or if we should only trust deterministic consistency and integrity thresholds
More from coding & agent
- GPT-6 Astra beats Factorio with enemies in 44 in-game hours at ~$4,500 API cost — liminal_bardo · 2026-09-11
- Treating agents like 50 First Dates: a 3-layer context system so every conversation doesn't start from zero — evielync · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11
- ARRM targets silent economic regressions in AI agents that functional tests miss — Beautiful_Belt_601 · 2026-09-11
- Dev builds browser 3D pizza delivery game with Claude: physics, GPS pathfinding, traffic AI — vinishkapoor · 2026-09-11
- Build X Carousel Posts from One Wide Image: A Splitter Tool Plus YouMind Skill Workflow — sujingshen · 2026-09-11