Build-A-Chip-Bench: end-to-end chip design benchmark, top agent scores just 16.6%
teortaxesTex · x · 2026-10-29
AlphaDesignAI, with NVIDIA contributions, released Build-A-Chip-Bench, evaluating AI agents across the full chip design flow from workload specification to GDS layout, using open-source tools and an open PDK. Scoring combines functional verification, clean GDS pass and PPA-vs-constraint.
- Claude Fable 5.1 leads at 16.6% (65% functional pass, 75% GDS pass)
- GPT-6 Astra scores 8.7% despite the highest pass rates (85% functional, 85% GDS), limited by PPA
- Gemini 3.8 Flash: 0.8%; Kimi K3: 0.2%; GLM-5.3: 0.0%
The takeaway: generating correct RTL is one milestone; delivering a physically implementable design remains far out of reach for frontier models.
More from coding & agent
- Anthropic says no regression behind Claude Code usage complaints, blames heavy sessions — bcherny · 2026-10-29
- LlamaIndex launches Document Parsing Protocol: 140 rules to standardize OCR output for agents — llama_index · 2026-10-29
- exe.dev Ships WIF Support: Agents Make OAuth Token Exchange Finally Usable for All — davidcrawshaw · 2026-10-29
- Protecting action-taking agents from prompt injection: move defense out of the model layer — Informal-Dust4499 · 2026-10-29
- Designing agent approval screens: show the effect, not the narrative — ClickOk5811 · 2026-10-29
- "Babysit this for me" is now the number one request people make to AI agents — RichardsonDx · 2026-10-29