Running Muse Glimmer 30B on RX 7600 XT 16GB: Hits 20 t/s with Speculative Decoding
DanC403 · reddit · 2026-08-12
A developer shared their experience running the Muse Glimmer 30B model locally on an entry-level AMD Radeon RX 7600 XT (16GB) GPU.
- Hardware & Environment: Host running Ryzen 5 4600G with 96GB DDR4 RAM. The GPU is passed through to a Debian Sid VM via QEMU/KVM, utilizing ROCm 7.2 compiled with llama.cpp.
- Quantization & Acceleration: Uses the UD-Q2-K-XL quantization paired with DFlash speculative decoding to boost throughput.
- Performance: With a 62k context size, prompt evaluation hits 308 t/s, and generation speed reaches 20 t/s.
- Coding Test: Fed 8 JavaScript files and 1 HTML file into the model. It successfully outputted functional code diffs on the first try.
Related event: Meta's Open-Source Muse Glimmer 30B Impresses in On-Device Agent Tests(14 posts)→
More from Infra
- Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16
- Meme: driving to the chip factory in 2028 captures AI's compute anxiety — beffjezos · 2026-10-16
- One volunteer maintains the time zone file behind 4 billion phones — danbri · 2026-10-16
- DeepSeek V4.1 Flash on 4x DGX Spark: weights alone are 511GB, the 494 tok/s claim doesn't fit — Aroochacha · 2026-10-16
- Open-sourced Intel GPU Vulkan inference upgrades: 2x ingestion, 60%+ faster inference — Fun-Aardvark-1143 · 2026-10-16
- ELSFP pain points: >3dB insertion loss, costly PMF, harsh packaging requirements — jwt0625 · 2026-10-16