600 tok/s single-request on Qwen 35B with Ninfer on an RTX Pro 6000
CharlesStross · reddit · 2026-09-18
A Reddit user reports 600 tok/s on a single request running Qwen3.6 35BA3B with Ninfer on an RTX Pro 6000. Even if it uses 20x more tokens, it's still faster than many local models for read-and-find or brute-force coding tasks — 'not quite Cerebras' but a fun high-throughput local setup.
More from Infra
- Fire at Taiwan Panasonic PCB materials plant may tighten CCL supply further — zephyr_z9 · 2026-11-04
- Same Claude model served by Bedrock one day and Anthropic the next, dev finds — georgos_pappas · 2026-11-04
- Self-improving software is inevitable: closing the agent-observability loop — bibryam · 2026-11-04
- Gigabyte's AI TOP motherboard packs 2TB RAM and 9 GPUs for home AI labs — vincent_koc · 2026-11-04
- Building a 7-GPU 132GB VRAM local inference rig: consolidate on EPYC or split across two systems? — infectiousstupidity · 2026-11-04
- imec neural probe integrates photonics directly into 130nm CMOS backend — jwt0625 · 2026-11-04