TokenSpeed open-source inference engine hits 2.8k stars with Day 0 support and spec decoding
zhyncs42 · x · 2026-10-23
LightSeek open-sourced TokenSpeed, a "speed-of-light" LLM inference engine with 2.8k GitHub stars. The team reports months of Day 0 support and inference-optimization collaboration with frontier open-source model labs, plus a full speculative decoding stack spanning algorithm research, draft-model training with TorchSpec, and efficient serving. It ships kernels for AMD/NPU/Blackwell hardware and supports DeepSeek, GLM, Qwen, Kimi, MiniMax and more under MIT license, arguing inference infrastructure must be open.
More from Infra
- Meta researcher: horrifying infra, not talent, is why OSS can't catch closed frontier — A_K_Nain · 2026-10-23
- Jane Street and NVIDIA speed up Mamba-2 with custom cuTile kernels — blelbach · 2026-10-23
- LatentLoop: asymmetric compute allocation cuts looped-MoE training cost by 19% — pmttyji · 2026-10-23
- Same AI request emits 8.5x more CO2 in Texas than in Sweden — jlundberg · 2026-10-23
- SK Hynix Pulls Plug on Solidigm US IPO as Chip Stocks Slide, Shares Down ~30% This Quarter — rohanpaul_ai · 2026-10-23
- Designer mocks Nvidia: buy idle GPUs or lose access to new ones—definitely not a bubble — michalmalewicz · 2026-10-23