OpenAI hardware VP details first custom chip Jalapeño and its nine-month tape-out
bigdata · x · 2026-09-20
The Data Exchange interviews Richard Ho, VP of Hardware at OpenAI, about Jalapeño, the company's first custom AI accelerator:
- Why custom: owning more of the stack to cut inference cost; unusual combo of high throughput and low latency, designed for general-purpose and open-source LLMs.
- Architecture: blank-slate design bringing memory and compute closer; prefill/decode, AI-optimized kernels, speculative decoding.
- AI-accelerated engineering: parts of the cycle compressed to a nine-month tape-out; roadmap reaches Gen 3 with HBM4 and new memory.
- Also covered: CUDA's moat, power becoming the datacenter bottleneck, RL/agents driving inference demand, InferenceX/AgentX benchmarks.
Related event: OpenAI Hardware VP Details Jalapeño, Its First In-House AI Chip(2 posts)→
More from Infra
- Agent Substrate Calculator Estimates Session Resume Speed Under Load — pjausovec · 2026-11-12
- Dell expands AI Data Platform: agents have a data-context problem, not a model problem — DavidLinthicum · 2026-11-12
- Potion UI's local login system turns your PC into a multi-user AI server — cocktailpeanut · 2026-11-12
- Recursive's Eureka Machine ships Flint: a sparse 40B model with 2.5B active for local agentic coding — agihouse_org · 2026-11-12
- AWS Finally Built Its Own AWS Emulator — super_linear · 2026-11-12
- Megakernel is multiprogramming for GPUs: 'GPUs are entering the CPU age' — pranjalssh · 2026-11-12