YASAKANI inference engine claims 1.69–2.86x speed wins over SGLang, vLLM and llama.cpp on RTX 5090
udmrzn · x · 2026-11-01
Developer @yasakanidev unveiled YASAKANI, a self-proclaimed world's fastest inference engine, with benchmark data on RTX 5090 running Qwen3.8-27B:
- Speed: 380.6 tok/s median for reasoning generation vs SGLang 183.9, vLLM 182.9, llama.cpp 146.9 — fastest in all 10 tests, 1.69–2.86x faster than rivals' best configs (DFlash 2/DSpark/MTP enabled), taking the slower of two runs. Also 2.39x faster than ExLlamaV3 (MTP). TensorRT-LLM couldn't run MTP on RTX 5090, so not compared.
- Compression: proprietary .mgtm format shrinks the model to 12.4GB (vs 17.4GB for GGUF Q4KM), with roughly half the KLD divergence from the original model vs the best same-size public GGUF (0.021 vs 0.039), and better than GSQ-RCO (IQ3S) at 0.044.
The project will be open-sourced for free, with release dates to be announced on the account.
More from Infra
- Musk: Colossus 2 to host ~1.2M Blackwell GPUs by end of year — scaling01 · 2026-11-01
- Local Maximus meme: six RTX 3090s, 256GB RAM, servant to no API — gklambauer · 2026-11-01
- Tokenomos open-sourced: rewritten vLLM frontend makes small-model inference ~3x faster on RTX 5090 — bdsqlsz · 2026-11-01
- DeepSeek Details Always-On Agent Design With External Watchdog Session for Progress Monitoring — teortaxesTex · 2026-11-01
- MegaMoE kernel posts 4x per-GPU throughput vs baseline on 8xH200 DeepSeek V4.1 Flash — michellechen · 2026-11-01
- Starcloud hosts first factory tour of its new space datacenter facility — Scobleizer · 2026-11-01