OpenAI Unveils Jalapeño, Its First Custom Inference ASIC, at Hot Chips
OpenAI's chip team presented the architecture of its custom inference ASIC, codenamed Jalapeño, at Hot Chips—the company's first architecture-level disclosure of its in-house silicon. According to attendees and leaks, the chip targets inference, uses InferenceX and open-source models as its North Star benchmarks, and is estimated to deliver B200-class throughput at lower power, drawing wide industry attention.
Confirmed
- Jalapeño is an inference-focused ASIC shown by OpenAI at Hot Chips, with InferenceX and open-source models as benchmark directions.
- Each chip has its own dedicated HBM slice, with cores communicating directly over an on-chip network.
- The most notable architectural feature is separating Speculate and Decode into distinct stages.
- BenBajarin summarized the design philosophy as "No show flops, all achievable flops"—no demo-only compute, only realistically attainable performance.
- appenz estimated throughput equivalent to B200 with lower power consumption and higher efficiency at high token rates.
- OpenAI also shared engineering details: tape-out from RTL in just 9 months, and an AI optimization loop that improved a low-efficiency kernel from 0.31% to 88.94% of the physical limit within 2 days, producing code better than top engineers'.
Unconfirmed
- Leaker beffjezos claimed 128 Jalapeño chips per rack and 2,048 per pod, with tensor parallelism and expert parallelism using different interconnects; these scaling details are unverified.
Why it matters
- Jalapeño marks a key step in OpenAI reducing dependence on third-party GPUs. The speculation/decode separation and all-achievable-flops orientation show targeted optimization for LLM inference throughput and efficiency.
- The 9-month tape-out and AI code-optimization loop demonstrate extreme efficiency in AI-assisted chip design, which could reshape both inference compute economics and chip R&D paradigms if scaled.
2026-08-26 ~ 2026-08-28 · 7 related posts
- Episode 1: OpenAI unveils first in-house inference chip Jalapeño with benchmark claims beating Nvidia(2026-08-25, 67 posts)
- Episode 2: OpenAI Unveils Jalapeño, Its First Custom Inference ASIC, at Hot Chips(2026-08-26, 7 posts)
- Episode 3: OpenAI Reveals First Real Data of Its Inference Chip Jalapeño, Sparking Debate Over Nvidia's Weakness(2026-08-26, 6 posts)
- Episode 4: OpenAI Used AI Models to Help Design Its First Chip(2026-08-27, 2 posts)
- Episode 5: Report: OpenAI's in-house Jalapeño chip may outperform Nvidia Blackwell(2026-08-27, 2 posts)
- Episode 6: OpenAI's Custom Inference Chip Jalapeño Tapes Out, Claiming Efficiency Gains Over Nvidia(2026-08-30, 14 posts)
- Episode 7: OpenAI Unveils First In-House Inference Chip Jalapeño with Broadcom(2026-09-04, 2 posts)
Primary sources
- [source] OpenAI reveals 'Jalapeno' inference ASIC at Hot Chips — beffjezos · 2026-08-26
- OpenAI Jalapeno Chip Details: Dedicated HBM per Chip — beffjezos · 2026-08-26
- OpenAI Jalapeno scale-out leak: 128 chips per rack, 2048 per pod — beffjezos · 2026-08-26
- [source] OpenAI Chip Insight: No Show Flops, All Achievable Flops — BenBajarin · 2026-08-26
- OpenAI reveals Jalapeno chip: B200-level throughput with novel speculate-decode split — appenz · 2026-08-26
- OpenAI's chip engineering: 9 months to tapeout, AI beats humans — kleffew94 · 2026-08-28
1 near-duplicate retellings: appenz