NVIDIA-backed chip design bench: DeepSeek V4.1F matches Claude Opus 5 on CVDP pass@3
teortaxesTex · x · 2026-10-29
Build-A-Chip-Bench evaluates AI agents through the full chip-design flow from workload spec to GDS layout, using open-source tools and PDK, with contributions from NVIDIA. Key points: DeepSeek V4.1F ranks equal to Claude Opus 5 (pass@3) on NVIDIA's CVDP hardware verification eval; CVDP has plateaued around 65-70% and the newer ChipBench has passed 90%, so these benchmarks are saturating. Red flag: all models were run with OpenCode as the agent, so methodology needs polish. Point tests miss downstream issues (timing, area, routing) that only appear across the full flow — the benchmark's core motivation.
More from Models
- Researchers steer four neurons in Qwen3.5-4B to turn it into a catgirl named Meecot — burny_tech · 2026-10-29
- Anthropic says no regression behind Claude Code usage complaints, blames heavy sessions — bcherny · 2026-10-29
- OpenAI: Codex and Chat capabilities will merge this year, unified limits feared — CtrlAltDwayne · 2026-10-29
- Community reverse-engineers Qwen3.5-27B from public weights: it's grown from the 9B model — davidad · 2026-10-29
- Reddit User: AI Labs' Tier Sprawl Is Broken — One Model, One Price Would Win — Tight-Grocery9053 · 2026-10-29
- Mystery model 'Kerria' spotted on LMArena, tokenizer errors suggest it could be Qwen's next model — lyraxana · 2026-10-29