OpenAI's Custom Inference Chip Jalapeño Tapes Out, Claiming Efficiency Gains Over Nvidia

According to SemiAnalysis, OpenAI's custom inference chip Jalapeño, built with Broadcom, has taped out. It targets inference rather than training, aiming to cut ChatGPT serving costs, speed up responses, and reduce dependence on Nvidia, with deployment planned before year's end.

Confirmed

• OpenAI reported that on GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T, peak-throughput tokens per watt improved 1.5–1.9x over Nvidia GB200/GB300
• @rohanpaulai, citing SemiAnalysis: at 100 tok/s interactive speed, efficiency is 11M tok/s/MW, roughly 2x comparable Blackwell configurations; the chip went from design to tape-out in 16 months and beat Nvidia on multiple open models
• @prasannasays relayed OpenAI's Hot Chips '26 disclosure: the design faced a late-stage floorplanning crisis; the team adopted the XLS hardware programming environment, with over half the cores written in it
• Analysts (@BenBajarin, @thehiphopswami, @akbirthko) published deep dives on Nvidia GPU inference shortcomings (E2E latency, RoCE networking, DPU bottlenecks, KV cache cost) and inference-optimized design ideas such as Core Slice architecture

Unconfirmed

• Tape-out timeline accounts conflict: @thehiphopswami cited 9 months and 'AI-designed', while the SemiAnalysis report says 16 months; no official confirmation
• All efficiency figures are OpenAI/SemiAnalysis claims lacking independent third-party verification

Why it matters

• Analysis relayed by @AccBalanced argues Jalapeño inverts Nvidia's 'can programmers use it' principle by assuming AI writes the compilers, freeing the architecture from human-usability constraints
• The XLS practice at Hot Chips corroborates this: AI is reshaping the chip design workflow itself
• If the efficiency claims hold, it directly threatens Nvidia's inference-chip position and solidifies the custom-silicon trend among top AI labs

2026-08-30 ~ 2026-08-31 · 14 related posts

Full story(7 episodes)→

• Episode 1: OpenAI unveils first in-house inference chip Jalapeño with benchmark claims beating Nvidia(2026-08-25, 67 posts)
• Episode 2: OpenAI Unveils Jalapeño, Its First Custom Inference ASIC, at Hot Chips(2026-08-26, 7 posts)
• Episode 3: OpenAI Reveals First Real Data of Its Inference Chip Jalapeño, Sparking Debate Over Nvidia's Weakness(2026-08-26, 6 posts)
• Episode 4: OpenAI Used AI Models to Help Design Its First Chip(2026-08-27, 2 posts)
• Episode 5: Report: OpenAI's in-house Jalapeño chip may outperform Nvidia Blackwell(2026-08-27, 2 posts)
• Episode 6: OpenAI's Custom Inference Chip Jalapeño Tapes Out, Claiming Efficiency Gains Over Nvidia(2026-08-30, 14 posts)
• Episode 7: OpenAI Unveils First In-House Inference Chip Jalapeño with Broadcom(2026-09-04, 2 posts)

• [source] OpenAI's Jalapeño Chip Beats Nvidia GB200 in Efficiency — Beth_Kindig · 2026-08-30
• OpenAI's Secret Weapon: AI-Designed 'Jalapeno' Chip Taped Out — thehiphopswami · 2026-08-30
• From Nvidia's Flaws to OpenAI Jalapeño: A Deep Dive into Redesigning Inference Chips — thehiphopswami · 2026-08-30
• OpenAI's Jalapeño inverts NVIDIA's chip design principle for AI compilers — AccBalanced · 2026-08-30
• OpenAI's Jalapeño inference chip beats Blackwell by up to 1.9x per watt — shashib · 2026-08-30
• [source] OpenAI's Jalapeño Chip Beats Nvidia Blackwell in Efficiency — rohanpaul_ai · 2026-08-30
• [source] Deep Dive: OpenAI's Jalapeño Chip Outperforms Competitors — rohanpaul_ai · 2026-08-30
• OpenAI used AI to write over half of its custom chip cores in Rust-like XLS language — prasanna_says · 2026-08-31
• OpenAI Unveils Jalapeño, a Custom Inference Chip Built With Broadcom to Cut ChatGPT Costs — thione · 2026-08-31

5 near-duplicate retellings: Beth_Kindig · thehiphopswami · BenBajarin · akbirthko · AccBalanced