Forked PolyStrata hits ~1000 tok/s prefill for small model on RX 6700 XT
ireallydontcare00 · reddit · 2026-11-23
The author forked the PolyStrata inference engine and tuned it for Inkling-2-Mini, benchmarking the UD-IQ4XS GGUF on an RDNA2-based RX 6700 XT:
- Prefill: 992 tok/s at 2K context, 780 tok/s at 32K, 628 tok/s at 65K
- Decode: 46.5–49.8 tok/s for short agent requests over HTTP; 41.0 tok/s for 128 output tokens after a 16K prompt
No quality loss was reported, and the code is open-sourced for anyone to try.
More from Infra
- MLX bug fix nearly triples decode speed on M1/M2 Ultra Mac Studio — SumitGup · 2026-11-23
- Inference practitioner on GPU co-ops: saturation math, weight caching, and quantization as dark art — sloppenheimer · 2026-11-23
- Nvidia's tale of two weeks: record revenue alongside regulatory probes and governance drama — emmanuelvivier · 2026-11-23
- Oracle raises $20B ($10B equity, $10B bonds) to fund OpenAI infrastructure deal — emmanuelvivier · 2026-11-23
- DeepSeek on 4 DGX Sparks codes great with Opus 5.5 planning, GLM-5.3 not ready — AIFlow_ML · 2026-11-23
- Thinking Machines open-sources looped model variant matching mini on coding benchmarks — SonglinYang4 · 2026-11-23