Strata EXL3 AMD fork gains 10-20% tok/s, runs 3bpw at IQ4-level quality
nasone32 · reddit · 2026-11-13
Reddit user nasone32 updated their Strata EXL3 fork for AMD, with reworked HIP kernels and memory management delivering 10-20% faster decode (up to 24% in some cases), plus experimental MiMo V2.6 Flash support at 25-30 tok/s on two cards (2.20 bpw MOPD quant).
Key point: EXL3 bpw isn't comparable to GGUF bit counts — 3.05 bpw EXL3 lands roughly at IQ4KXS quality and 4.05 bpw at Q5KM, so you get 4-bit-class quality at near-3-bit speeds. The update adds optional expert-weight repacking into a CPU-friendlier layout for RAM-based expert offload; beneficial on single-GPU setups, usually not on two.
Benchmarks on a Ryzen 9 7950X + 2x RX 7900 XTX (ROCm): 3.05 bpw hits 87.33 tok/s on CODE (+11.6%) with two GPUs; 4.05 bpw single-GPU with CPU assist reaches 55-60 tok/s at Q5KM quality. 32K prefill is unchanged (1,500 tok/s dual-GPU).
More from Infra
- 4x R9700 EXL3 serving: 1,094 tok/s Qwen decode, all optimization tricks open-sourced — mikoskinen · 2026-11-13
- Sweden's all-wood EcoDataCenter: 1.15 PUE, just 1.17g CO2/kWh, zero water use — filwi · 2026-11-13
- Swapping one month of Scaleway bills for a 16GB ThinkCentre to run agents locally — jiayuan_jy · 2026-11-13
- From reform-school toilets to $5T: a timeline of Jensen Huang and NVIDIA — tibo_maker · 2026-11-13
- TSMC discloses hack in SEC filing, details still scarce — gmays · 2026-11-13
- MI50 32GB vs V100 32GB for local inference: MI50 costs ~1.5x less — Gold-Produce9591 · 2026-11-13