AMD open-sources TokenSpeed-Kernel: portable LLM inference kernels across GPUs
zhyncs42 · x · 2026-10-20
AMD presented and open-sourced TokenSpeed-Kernel at the Triton Developer Conference, alongside talks on Triton Gluon performance on MI450/MI455X and rack-scale symmetric communication on AMD Helios.
Key points:
- TokenSpeed-Kernel is a standalone open-source subsystem tackling backend complexity in LLM inference: a layered API and registry system decouple the high-level runtime from hardware-specific code.
- Platform-agnostic interfaces: the runtime (owning model execution, scheduling metadata, page tables, routing state) calls the same public APIs, with pluggable kernels delivering performance on AMD and NVIDIA.
- Demonstrated with GPT-OSS: AMD GPT-OSS 120B reaches top-tier performance using Gluon kernels, showing the layering doesn't trade away backend speed.
More from Infra
- Diamond-embedded copper vias shown to boost chip thermal conductivity — jwt0625 · 2026-10-20
- Stealth startup Physis debuts world foundation model play with PhysisServe, core tech headed to vLLM-Omni — vllm_project · 2026-10-20
- Barcelona Supercomputing Center AI Factory Accelerator Names Three AI Champions — NVIDIA Blog · 2026-10-20
- Luma's MARBLE: exact sequence parallelism for packed multimodal masks, wins 47/48 vs FlashMask+Ulysses — mrjonfinger · 2026-10-20
- Dual R9700 layer-split side effect: per-card load under 50%, fans stay quiet — yahbluez · 2026-10-20
- PyTorch Conference poster to break down KV cache offloading: when CPU beats shared storage — PyTorch · 2026-10-20