OpenAI's VP of Hardware on Jalapeño: first custom AI chip taped out in nine months
ymatias · x · 2026-09-18
OpenAI VP of Hardware Richard Ho detailed the company's first custom AI accelerator Jalapeño on The Data Exchange podcast: AI tools helped compress parts of the development cycle to just nine months to tape-out. Key points:
- Why custom silicon: owning more of the stack to cut inference costs; a blank-slate architecture bringing memory closer to compute, breaking the throughput-vs-latency trade-off, targeting general-purpose LLMs and open-source models.
- Software stack: whether CUDA is still a moat, Triton, prefill/decode optimization, AI-optimized kernels, speculative decoding, plus InferenceX and AgentX benchmarks.
- Roadmap: Gen 1 to Gen 3 with HBM4 and new memory; he argues power, not chips, is becoming the data center bottleneck.
- AI-assisted engineering: increasingly capable AI tools could make expert engineering teams dramatically more productive.
Related event: OpenAI's first in-house AI chip Jalapeño taped out in 9 months(5 posts)→
More from Infra
- Cloudflare Talk: Swapping Some Agent LLM Calls for Fast Models Like Jev Gives Big Speedups — michellechen · 2026-11-12
- Cloudflare Trains Its Own Clef and Clairic Models to Complete Its Agent Stack — michellechen · 2026-11-12
- AWS open-sources 23 production-ready GenAI assets, cutting build time from months to weeks — AWS ML Blog · 2026-11-12
- Sixth Street CEO: ~$200B of Chips Sitting in Warehouses as Data Centers Lag Behind — SumitGup · 2026-11-12
- AWS CEO Matt Garman: Space Data Centers Are 10-20 Years Away, Likely Not Economically Feasible — rohanpaul_ai · 2026-11-12
- CoreWeave Database Goes GA: Managed PostgreSQL Running Inside Your CKS Cluster Next to GPUs — _ScottCondron · 2026-11-12