Photon 2.6 ships FP8 + speculative decoding, runs Qwen3.5 27B at 400+ tok/s on B200
Bedrovelsen · x · 2026-09-30
Photon 2.6 is out with FP8 inference and speculative decoding support. The author demonstrates running Qwen3.5 27B at over 400 tokens per second on an NVIDIA B200, showcasing the throughput of the latest inference stack on flagship accelerators.
More from Infra
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16
- Starlink Now Protects Greek Trains From Collisions With 5cm-Accurate Tracking of 134 Locomotives — XFreeze · 2026-11-16
- Google donates Agent Substrate, gVisor and OpenRL to CNCF; Agent Substrate 1.0 goes GA on GKE — dl_weekly · 2026-11-16
- Nvidia scandal isn't slowing down: Huang's near-$100B power web under strain — The Verge AI · 2026-11-16