Flint on RTX PRO 6000 in vLLM: beats official scores, full config shared
DataCraftsman · reddit · 2026-10-22
A Reddit user ran Flint (NVFP4) in vLLM on an RTX PRO 6000 and shared full benchmarks and configs: sampled LiveCodeBench v6 score of 82.2 (official: 80.8) and SWE-bench Verified of 80 (official: 74.1); 110-180 tok/s single stream with 2,500 tok/s peaks at 64 concurrent requests. The post includes a complete docker compose for vLLM (262K context, glm47 parsers, prefix caching) plus a chattemplate.jinja fix, and the author plans an A/B against qwen3.8 27b while hoping someone adds vision support.
More from Infra
- Cadence announces sovereign AI cloud for European chip designers — karlfreund · 2026-10-23
- SGLang Deep-Dive Blog Series Breaks Down KV Cache, Posts up to 15.5x Throughput Gains — BanghuaZ · 2026-10-23
- DynaLM: Open-Source CPU-First LLM Inference Engine Seeks Feedback — Unlikely_Safety_7456 · 2026-10-23
- Capital Is Now the AI Buildout Bottleneck, Says Analyst Citing Jensen Huang — JOBhakdi · 2026-10-23
- DynaLM: an open-source CPU-first LLM inference engine seeks testers and feedback — Unlikely_Safety_7456 · 2026-10-23
- Cerebras explains how it deploys trillion-parameter models on wafer-scale chips — bookwormengr · 2026-10-23