FULL STORY

OpenAI's Jalapeño: From Tape-out to Benchmarks

OpenAI's first in-house inference chip Jalapeño, built with Broadcom, went from leaked tape-out reports to a full Hot Chips reveal, with benchmark claims beating Nvidia's GB200/GB300 sparking debate over Nvidia's inference shortcomings.

2026-08-25 ~ 2026-09-04 · 7 episodes · 100 posts

Episode 1 · OpenAI unveils first in-house inference chip Jalapeño with benchmark claims beating Nvidia (2026-08-25, 67 posts)

OpenAI published the first benchmark results for its in-house inference chip Jalapeño, claiming wins over Nvidia's GB200 and GB300 flagship systems; SemiAnalysis's teardown similarly views it as a general-purpose LLM inference ASIC. Co-developed with Broadcom in roughly 16 months from team formation to tape-out, it is slated for deployment by year-end as the first step of a multi-generation roadmap. Current caveat: the striking numbers come from OpenAI's own InferenceX benchmark and SemiAnalysis analysis, with no independent third-party or Nvidia verification yet.

Confirmed

  • OpenAI officially announced Jalapeño with first test results; the chip was co-developed with Broadcom in about 16 months (@OpenAI, @dylan522p, @zephyrz9, @borowcy).
  • On OpenAI's InferenceX public benchmark using GPT-OSS 120B, DeepSeek R1 670B and Kimi K2: 1.5–1.9x higher work per watt, 1.7–3.6x lower end-to-end latency, higher peak throughput per kW, lower token latency, and better TCO than compared commercial systems (@OpenAI, @kimmonismus, @Polymarket).
  • At matched DeepSeek R1 decode speed, throughput per kW is 104.3x that of Nvidia GB300 (12,258 vs 118 tok/s/kW) (@rohanpaulai).
  • SemiAnalysis's teardown says Jalapeño is a from-scratch general-purpose LLM inference ASIC, not narrowly tuned to OpenAI models, with measured efficiency exceeding Nvidia and AMD Blackwell-class parts; per-watt performance up to 2x above Nvidia's Rubin announced in July, even without speculative decoding (@zephyrz9, @dylan522p, @gddr5, @nickbaumann).
  • OpenAI says the chip will speed up ChatGPT responses, Codex sessions and agent responsiveness, and ensure reliable access as demand grows (@OpenAI).
  • Deployment is planned before year-end; a second generation is in deep development and a third is taking shape (@OpenAI).
  • Reports relayed by @thesaraharminta put power at 700W, designed for large-model inference to cut data-center costs.
  • Cathie Wood, citing ARK analysis, says Broadcom is helping OpenAI build the chip, with Jalapeño at the throughput-per-kW frontier across decode speeds on GPT-OSS-120B (@CathieDWood).

Unconfirmed

  • All performance figures lack independent verification; test conditions behind the '104.3x' figure (system configs, quantization, batching) await further disclosure.
  • @WonderfulBuffalo32 says OpenAI claims wins over the newer Vera Rubin platform, while @thesaraharminta's relayed report says internal tests did not compare against Vera Rubin — a discrepancy pending clarification.
  • Speculation relayed by @cloneofsimo that the chip merges GPU/TPU strengths, won't be sold externally, and that China will clone it within three years is unverified opinion.

Why it matters

  • This is OpenAI's first public silicon benchmark, marking its strategy to reduce Nvidia dependence entering execution; if the general-purpose ASIC positioning holds, it poses real competitive pressure on Nvidia and AMD in data-center inference.
  • The disclosed second- and third-generation roadmap signals a long-term silicon product line rather than a one-off experiment, potentially reshaping the cloud inference chip landscape.

47 more related posts →

Episode 2 · OpenAI Unveils Jalapeño, Its First Custom Inference ASIC, at Hot Chips (2026-08-26, 7 posts)

OpenAI's chip team presented the architecture of its custom inference ASIC, codenamed Jalapeño, at Hot Chips—the company's first architecture-level disclosure of its in-house silicon. According to attendees and leaks, the chip targets inference, uses InferenceX and open-source models as its North Star benchmarks, and is estimated to deliver B200-class throughput at lower power, drawing wide industry attention.

Confirmed

  • Jalapeño is an inference-focused ASIC shown by OpenAI at Hot Chips, with InferenceX and open-source models as benchmark directions.
  • Each chip has its own dedicated HBM slice, with cores communicating directly over an on-chip network.
  • The most notable architectural feature is separating Speculate and Decode into distinct stages.
  • BenBajarin summarized the design philosophy as "No show flops, all achievable flops"—no demo-only compute, only realistically attainable performance.
  • appenz estimated throughput equivalent to B200 with lower power consumption and higher efficiency at high token rates.
  • OpenAI also shared engineering details: tape-out from RTL in just 9 months, and an AI optimization loop that improved a low-efficiency kernel from 0.31% to 88.94% of the physical limit within 2 days, producing code better than top engineers'.

Unconfirmed

  • Leaker beffjezos claimed 128 Jalapeño chips per rack and 2,048 per pod, with tensor parallelism and expert parallelism using different interconnects; these scaling details are unverified.

Why it matters

  • Jalapeño marks a key step in OpenAI reducing dependence on third-party GPUs. The speculation/decode separation and all-achievable-flops orientation show targeted optimization for LLM inference throughput and efficiency.
  • The 9-month tape-out and AI code-optimization loop demonstrate extreme efficiency in AI-assisted chip design, which could reshape both inference compute economics and chip R&D paradigms if scaled.

Episode 3 · OpenAI Reveals First Real Data of Its Inference Chip Jalapeño, Sparking Debate Over Nvidia's Weakness (2026-08-26, 6 posts)

At Hot Chips, OpenAI published the first real-world data for its in-house inference chip Jalapeño, triggering wide debate: supporters argue it exposes Nvidia's architectural weaknesses in inference, while skeptics question the comparison baseline and volume prospects.

Confirmed

  • OpenAI's data shows Jalapeño delivering 1.5–1.9x AI performance per watt versus the compared system at peak throughput, with 1.7–3.6x lower end-to-end latency (@ArtisticPhone9367, who also complained the documentation is hard to read).
  • Architecturally, it uses a NUMA-style local HBM slice design to address operand latency, and AI-assisted EDA took it from RTL to tape-out in about 9 months (SemiAnalysis podcast, relayed by @thehiphopswami).
  • The podcast argues "dark silicon is cheaper than idle accelerators," and one balanced chip could beat a GPU + LPU combo.
  • @beffjezos holds that Nvidia's microarchitecture was not designed for inference and is suboptimal in multiple ways; its moat lies in CoWoS packaging and HBM supply-chain dominance rather than microarchitecture.

Unconfirmed

  • Analysis relayed by @AccBalanced notes OpenAI compared against an older Blackwell system using HBM3e rather than an HBM-matched configuration, casting doubt on "beats Nvidia" headlines.
  • @GavinSBaker (Gavin Baker) argues large-scale ultra-fast compute still needs GPUs and CS-4/5; Jalapeño struggles to compete with GPUs on financing rates and may face limited wafer allocation for two years.

Why it matters

  • These are the first public measured results from OpenAI's silicon effort; if the efficiency claims hold, they could unsettle expectations of Nvidia's inference dominance.
  • Yet the baseline dispute and capacity constraints suggest Jalapeño is, in the near term, more a tool for OpenAI to cut costs and strengthen bargaining leverage than a direct Nvidia replacement.

Episode 4 · OpenAI Used AI Models to Help Design Its First Chip (2026-08-27, 2 posts)

OpenAI revealed it built its first chip, codenamed Jalapeño, with AI models Sol and Astra assisting RTL design, dramatically accelerating the process; the chip is intended to run AI workloads.

Episode 5 · Report: OpenAI's in-house Jalapeño chip may outperform Nvidia Blackwell (2026-08-27, 2 posts)

SemiAnalysis founder Dylan claims OpenAI's first in-house chip 'Jalapeño' performs impressively and may outperform Nvidia's Blackwell and Rubin. The claim, which first circulated via YouTube, remains unverified.

Episode 6 · OpenAI's Custom Inference Chip Jalapeño Tapes Out, Claiming Efficiency Gains Over Nvidia (2026-08-30, 14 posts)

According to SemiAnalysis, OpenAI's custom inference chip Jalapeño, built with Broadcom, has taped out. It targets inference rather than training, aiming to cut ChatGPT serving costs, speed up responses, and reduce dependence on Nvidia, with deployment planned before year's end.

Confirmed

  • OpenAI reported that on GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T, peak-throughput tokens per watt improved 1.5–1.9x over Nvidia GB200/GB300
  • @rohanpaulai, citing SemiAnalysis: at 100 tok/s interactive speed, efficiency is 11M tok/s/MW, roughly 2x comparable Blackwell configurations; the chip went from design to tape-out in 16 months and beat Nvidia on multiple open models
  • @prasannasays relayed OpenAI's Hot Chips '26 disclosure: the design faced a late-stage floorplanning crisis; the team adopted the XLS hardware programming environment, with over half the cores written in it
  • Analysts (@BenBajarin, @thehiphopswami, @akbirthko) published deep dives on Nvidia GPU inference shortcomings (E2E latency, RoCE networking, DPU bottlenecks, KV cache cost) and inference-optimized design ideas such as Core Slice architecture

Unconfirmed

  • Tape-out timeline accounts conflict: @thehiphopswami cited 9 months and 'AI-designed', while the SemiAnalysis report says 16 months; no official confirmation
  • All efficiency figures are OpenAI/SemiAnalysis claims lacking independent third-party verification

Why it matters

  • Analysis relayed by @AccBalanced argues Jalapeño inverts Nvidia's 'can programmers use it' principle by assuming AI writes the compilers, freeing the architecture from human-usability constraints
  • The XLS practice at Hot Chips corroborates this: AI is reshaping the chip design workflow itself
  • If the efficiency claims hold, it directly threatens Nvidia's inference-chip position and solidifies the custom-silicon trend among top AI labs

Episode 7 · OpenAI Unveils First In-House Inference Chip Jalapeño with Broadcom (2026-09-04, 2 posts)

OpenAI unveiled Jalapeño, its first in-house inference processor co-developed with Broadcom, featuring 6 HBM4 stacks, 216GB memory and 15.4 TB/s bandwidth. The chip went from RTL freeze to tape-out in just 9 months, marking a key step in reducing reliance on NVIDIA.