Specter Speeds Up Single-GPU MoE Inference with Quantized Speculative Decoding

Shanghai Jiao Tong University proposed Specter, an inference system that tackles PCIe I/O bottlenecks when offloading MoE experts to host memory, using a lightweight quantized draft model and multi-step speculative decoding to achieve up to 1.88x speedup on a single GPU.

2026-10-21 ~ 2026-10-21 · 2 related posts

• Specter speeds single-GPU MoE inference with quantized multi-step speculative offloading — Wenfeng Wang · 2026-10-21
• Specter paper: quantization-enabled speculative offloading speeds up single-GPU MoE inference up to 1.88x — pmttyji · 2026-10-21