sFNX runs 27B dense at 114-300 tok/s on 3x V100 with P2P support
jfinkpottery · reddit · 2026-10-31
A Reddit user reports running the Qwen 27B dense model with sFNX on 3x V100 (Q8), hitting 114-300 tok/s depending on prompt, or 224 tok/s combined across 6 parallel requests. Key points:
- One user got flash-next working on unlocked 3090s, but consumer cards generally lack adequate P2P support without custom drivers and firmware
- For 1-2 cards, 1cat-vllm is just as good (sFNX refuses single-card setups), with slightly different speeds and quants
- With 3+ cards and good P2P speeds, sFNX is the only option found — and a significant jump over any 2-card setup
More from Infra
- SenseNova-U1.5-8B hits 125 tok/s on M5 Ultra in llama.cpp and spots a $1 receipt error — Binary_orchid · 2026-10-31
- Cloudflare Durable Objects now defer code updates up to 300s by default — irvinebroque · 2026-10-31
- Intel plunges 29%, falling below its 2000 dot-com era peak after 26 years — mitchdeg · 2026-10-31
- Clover-35B-A3B Quantized to 29.4 GB Runs at ~42 tok/s on a 64 GB Mac — viveky259 · 2026-10-31
- Liquid Agent brings fully local LFM3 agent to Linux with one-command install — JosephJacks_ · 2026-10-31
- A fully local office AI stack: MiniCPM, Qwen3, LFM2.5 and Whisper — HowDevelop · 2026-10-31