llama.cpp hits 1.2k t/s Qwen prefill on Strix Halo, matching closed-source Halogen
ilintar · reddit · 2026-09-13
A Reddit user pushed mainline llama.cpp's Qwen3.8 Flash Next prefill speed from 400 t/s to 1,200 t/s on AMD Strix Halo, matching closed-source server Halogen. They shipped a custom HIP runtime and install script, published an Opus-generated recap of the optimization journey, explained the mainline/fork ecosystem, and plan upstream PRs — work that should also benefit GLM 5.3 Flash's similar sparse attention.
More from Infra
- TSMC reportedly raising CoWoS packaging prices up to 20% next year on AI chip demand — Beth_Kindig · 2026-10-31
- effect-harness on Cloudflare: run agent loops, tools and credentials on separate VMs — samgoodwin89 · 2026-10-31
- Google's Project Suncatcher boots TPU in orbit and beams back first AI response — sundarpichai · 2026-10-31
- Together AI launches Post-Training platform for agentic RL with tool-use environments — togethercompute · 2026-10-31
- General Compute raises $100M for inference capacity as CoreWeave stock hits 12-month low — YvesMulkers · 2026-10-31
- Analyst warns of unstable AI compute feedback loop as Kioxia's NAND price warning triggers flash crash — bookwormengr · 2026-10-31