Standard Kernel uses AI-written kernels to make abandoned Intel Gaudi2 beat H100 by 1.6x on decode

anneouyang · x · 2026-11-19

Standard Kernel shows AI-generated software can make Intel's abandoned Gaudi2 outperform the NVIDIA H100: 5,409 tokens/s cached-prefill throughput (1.08x H100), 475 tokens/s decode (1.6x H100), and decode costs of $0.35 per 1M tokens — 9.4x cheaper than H100's $3.28. The takeaway: software, not hardware alone, determines accelerator performance and economics.

Original post →

More from Infra

Infra channel →