Unsloth Dynamic v3.1 UD-IQ1_M on RX 7900 XT: 53 tok/s with 192K context
zxspectrun · reddit · 2026-11-17
A detailed writeup of getting Unsloth Dynamic v3.1 UD-IQ1M running on the Strata RDNA3 fork for RX 7900 XT: 53 tok/s median decode, 192K context, 712 MiB VRAM to spare.
- Model registration took 40 lines of Python/JSON across setup.py, models.py, GGUF fingerprints and configs — no C++ or kernel rebuilds, since the fork already shipped HIP kernels for the IQ2/IQ4 expert types.
- Speed comes from tuning, not the model: the 68 GB model wants 48 GB RAM for experts, so the author details the full flag set — --expert-cache auto, int8 KV with 65536 resident, pinned-RAM budgeting, --ple-io direct, --prefill 16384, MTP spec decoding.
- They also ported RDNA-FAST's SSD conversation cache: an 85-line module with 7 hooks that snapshots conversations by prompt digest and only reads past the restored prefix.
More from Infra
- Baseten pushes DeepSeek V4.1 Flash to ~2,500 tok/s on Vera Rubin, 1.7x faster than GB300 — ying11231 · 2026-11-17
- Alchemy PR adds read-your-writes: clients auto-send command receipts with every read — samgoodwin89 · 2026-11-17
- Pipeshift claims 40% faster first token than Sarvam serving Gemma 4 31B at 150B tokens/day — NirantK · 2026-11-17
- LoopFill: training-free speculative decoding speeds up looped Transformers — UIUC-CS · 2026-11-17
- Tencent's DSpine speculative decoding ships GGUFs and a llama.cpp PR: 4.5x speedup on 27B — pmttyji · 2026-11-17
- Founder lands 1000 GPUs (40% GB300, 60% B300) on short-term contracts, dismissing the GPU shortage narrative — SumitGup · 2026-11-17