Hackathon project Screenshot to Tokens turns any screenshot into a full design system via SAM 3.1
usamawahabkhan · x · 2026-11-05
Open-source hackathon project "Screenshot to Tokens" (Meta Global AI Developer Hackathon) generates a full three-tier design system (primitive → semantic → component) from pixels for live apps with no design tokens.
Pipeline: Playwright captures a full-page screenshot plus computed styles; SAM 3.1 segments buttons, inputs, and icons from pixels even where markup describes nothing; Muse Spark names tokens with a reason each; a human reviews in an HTML report; hardcoded values in the real source become var(--token, original) so originals stay as fallbacks; four checks (zero-px fallback changes, invisible merged colors, reviewable diff, WCAG AA contrast) must pass before an automated PR opens. Only two steps use Meta models; the rest is deterministic code.
More from coding & agent
- Developer ships Drivr Glass: voice-first Grok cabin bot now displays cards in-car — bfrench · 2026-11-06
- Traveler says Grok bot planned and paid for his months-long Asia trip from just two context files — eptwts · 2026-11-06
- Aware lets you summon AI anywhere with @aware instead of a chat window — HaijunXia · 2026-11-06
- GitHub MCP server to replace server instructions with Agent Skills, aiming to save tokens — DanWahlin · 2026-11-06
- GitHub launches Copilot Fusion, a runtime router that auto-picks models and execution patterns — adnan_hashmi · 2026-11-06
- DIY AI Employee Factory: The 'Employee Zero' Template That Survives Crashes and Model Swaps — leebase65 · 2026-11-06