Tenstorrent hits 5,300 tokens/sec/user decoding gpt-oss-120b on just 8 Galaxy units
DavidBennett__ · x · 2026-11-21
Tenstorrent shared inference results: its Galaxy cluster decodes gpt-oss-120b at 5,300 tokens/sec/user.
- Each layer is split across a 4x4 mesh of 16 Blackhole chips
- Pipeline stages are reused while the next layer's weights load from DRAM into SRAM, delivering the speed of a 24-Galaxy deployment on just 8
- Japan-based inference company ai& announced Tenstorrent Wormhole and Blackhole Galaxies deployed across its Japanese data centers, with 100MW+ capacity spanning NVIDIA, AMD, Tenstorrent, and Rebellions chips, and a sovereign AI inference platform being provided to NEC
More from Infra
- Micron CEO: customer demand over next five years exceeds what we could commit — Beth_Kindig · 2026-11-21
- 2026 WebAI Summit Posts 25 Talks, 8 Per Week, Keynoted by Google's Jason Mayes — jason_mayes · 2026-11-21
- KV Cache Rules Everything: agent context re-reads cost you a lot, providers almost nothing — AccBalanced · 2026-11-21
- Darkbloom serves ~27B tokens a day on idle Macs worldwide, Gemma demand doubles — gajesh · 2026-11-21
- Open-source router factors prompt cache into LLM switching decisions for coding agents — Broad_Abies9390 · 2026-11-21
- Inkling 2 Mini Support Lands in llama.cpp via PR #32694 — No_Algae1753 · 2026-11-21