Jalapeño chip is just a systolic array with HBM: why custom inference silicon is easier than you think

ai · x · 2026-10-26

The Jalapeño chip is essentially a systolic array with lots of HBM — not crazy design, but squeezes far more performance from existing models.
Two startup directions: models/services that adapt workloads to exploit every corner of available hardware, and faster chip-design iteration workflows.
Frontier labs' custom chips are less impressive than assumed: inference-only, transformer-only (big MXU matmul units), kernels written near-ISA by RL-trained coding agents, no high-level compilers.
Building a truly general chip like an Nvidia GPU or Google TPU is much harder; expect many more inference chip companies as software cost collapses to token cost.

More from coding & agent