OpenAI's Jalapeño ASIC beats NVIDIA GB300 at half the power: 1.5-1.9x throughput per watt
新智元 · wechat · 2026-09-10
Anthropic has confirmed an internal custom silicon team for Claude inference chips, while OpenAI unveiled full benchmarks for its Jalapeño inference ASIC at HotChips: 700W vs NVIDIA's 1200W GB200 and 1400W GB300, with 1.5-1.9x higher throughput per watt and 1.7-3.6x lower end-to-end latency. The same week, Qualcomm signed a $60B custom inference chip deal with Amazon.
- Agents reshape compute: inference shifts from one-shot requests to long-running task chains (multi-turn reasoning, tool calls, retries), demanding always-on, recoverable, cost-controlled systems.
- Jalapeño's three-stage design: Prefill (compute-heavy), Draft (speculative sampling, latency-sensitive), Verification (bandwidth-heavy with bursty MoE traffic) on one chip, local KV cache, powering down idle units — the key to beating 1400W parts at 700W.
- China's gap: rich applications and diverse chip routes (RISC-V, Chiplet, compute-in-memory) but high interop/migration costs risk fragmentation.
- The piece ends promoting the AICC2026 conference in Beijing and a China AI compute report.
More from Infra
- Anthropic S-1: training infra edge cut per-task cost 40% and doubled revenue growth — BenBajarin · 2026-10-26
- Rewriting scaling law study: optimal synthetic data share rises from 10% to 32% with compute — heghbalz · 2026-10-26
- US states halt new data centers as 72% of Americans oppose local builds — YvesMulkers · 2026-10-26
- $5M Seed Round Can't Afford One DeepSeek Training Run: GPU-less AI Startup's Three Paths — FinanceYF5 · 2026-10-26
- HBM demand shifts: low-stack inference plans shelved, Rubin to go 12-high only — BenBajarin · 2026-10-26
- CircuitHub unveils The Grid, a factory-scale robot making PCBs in the US twice as fast as Shenzhen — garrytan · 2026-10-26