OpenAI's First Custom AI Chip Jalapeño Taped Out in Nine Months, Targets Low-Latency Inference
bigdata · x · 2026-09-18
Richard Ho, VP of Hardware at OpenAI, details Jalapeño, the company's first custom AI accelerator, in a podcast interview.
- Nine-month tape-out: AI tools helped compress parts of the development cycle, shrinking the design-to-tape-out timeline to nine months.
- Core motivation: Owning more of the hardware stack to lower inference costs and improve performance, rather than chasing peak compute.
- Architecture: A blank-slate design that brings memory and compute closer together, breaking the traditional throughput-vs-latency trade-off and targeting general-purpose LLMs and open-source models.
- Bottleneck: Power, not chips, is becoming the data center constraint.
- Software stack: CUDA, Triton, speculative decoding, kernel optimization, plus cross-model benchmarks like InferenceX and AgentX.
- Roadmap: From gen one to gen three, involving HBM4 and new memory; RL and agents are driving inference demand.
- CUDA moat: Discussion of whether CUDA remains a moat and the optionality created by AI and new hardware.
Related event: OpenAI's first in-house AI chip Jalapeño taped out in 9 months(5 posts)→
More from Infra
- Cloudflare Talk: Swapping Some Agent LLM Calls for Fast Models Like Jev Gives Big Speedups — michellechen · 2026-11-12
- Cloudflare Trains Its Own Clef and Clairic Models to Complete Its Agent Stack — michellechen · 2026-11-12
- Sixth Street CEO: ~$200B of Chips Sitting in Warehouses as Data Centers Lag Behind — SumitGup · 2026-11-12
- AWS CEO Matt Garman: Space Data Centers Are 10-20 Years Away, Likely Not Economically Feasible — rohanpaul_ai · 2026-11-12
- CoreWeave Database Goes GA: Managed PostgreSQL Running Inside Your CKS Cluster Next to GPUs — _ScottCondron · 2026-11-12
- micro1 scaled eval infra to 12× monthly tokens in 5 months, agent rollouts now start in 12 seconds — Exp_Mark · 2026-11-12