QuixiInfer: from-scratch Rust engine runs DeepSeek with 256k context on a single RTX 5090 at ~30 tok/s

QuixiAI · x · 2026-11-20

Developer QuixiAI released QuixiInfer, a from-scratch Rust inference engine that runs DeepSeek-v4.1-Flash-1017 on a single RTX 5090 with 256k context and up to 3 concurrent requests at 25-30 tok/s, using dynamic CPU and NVMe offloading (inspired by DeepInfer and Strata).

Benchmarks vs Strata:

The author is seeking contributors to test on 3090, 4090, and AMD GPUs.

Related event: Rust-Built QuixiInfer Runs DeepSeek MoE on a Single RTX 5090(3 posts)→

Original post →

More from Infra

Infra channel →