Swift-1.5-Qwen3.8-27B ported to hipfire MQ4: 2.2x faster end-to-end than GGUF on a 7900 XT
Queasy_Asparagus69 · reddit · 2026-10-23
A developer quantized the Swift-1.5 fine-tune of Qwen3.8-27B into hipfire's MQ4 format with an MTP head for speculative decoding, targeting AMD users who want more speed than llama.cpp.
Benchmarks (RX 7900 XT 20GB, ROCm 7.1, vs Q4KS GGUF):
- Prefill 1.4–1.85x faster; 1,100 tok/s at 16K context vs 700 on llama.cpp
- Decode: 36.5 tok/s on real prompts with MTP; 52–59 tok/s at 16–49K context
- Six real Rust coding tasks with 10–20K-token prompts: 237s vs 530s, 2.24x faster end-to-end
- 17.5 GB VRAM at 8K context; 49K+MTP is the recommended daily setting; layer offload verified at 62K (MTP) and up to 124K, though decode drops to 16–31 tok/s
Caveats: requires hipfire runtime v0.4.2.1+; llama.cpp can't load .mq4 files; only tested on gfx1100/7900 XT. Notably, the un-fine-tuned base model is slightly faster end-to-end — Swift's selling point is committing early rather than raw speed. Full details and run command on the HF model card.
More from Infra
- Black October: LiteLLM credential stealer, npm worm and thousands of poisoned GitHub workflows — AIFlow_ML · 2026-10-23
- July 28 MCP release makes remote servers stateless, biggest revision since launch — bibryam · 2026-10-23
- MCP keynote: AAIF hits 305 members, Skills over MCP and Tasks/Events roadmap revealed — bibryam · 2026-10-23
- Only 8% of cloud vendors support OIDC — dev calls to kill long-lived API keys — cramforce · 2026-10-23
- Intel CTO: chip output can at best 2X while AI compute grows 3X per year — SammaVaco · 2026-10-23
- Engineer reverse-engineers projector TV's 10s wake lag to a cold-booting Linux board — banteg · 2026-10-23