A 70B GGUF model stalls on an AMD R9700 as VRAM hits 30 GB but RAM stays flat
Developer-Y · reddit · 2026-07-28
A Reddit user is trying to run AstroSage 70B on an AMD Radeon AI Pro R9700 with ROCm 7.2 and llama.cpp, but the model stalls while loading.
- The GGUF file is a Q4KM build at roughly 42 GB.
- amd-smi shows about 30.3 GB of VRAM in use, while system RAM stays around 3.9 GB.
- The user is asking for advice from people who have successfully run a similar 70B setup on this hardware.
More from Infra
- MLA-based KV cache costs 12 GB per million tokens, with KDA state at 230 MB BF16 — zephyr_z9 · 2026-07-28
- Reddit debates the cheapest practical way to run K3 locally — ylchao · 2026-07-28
- K3 hit a usage-limit exploit on Higgsfield within a day of launch — Mediocre-Witness-778 · 2026-07-28
- TRELLIS.2 INT8 ConvRot now runs natively in ComfyUI on AMD ROCm — DrBearJ3w · 2026-07-28
- AI data-center buildout is reshaping grid incentives and reserve power — kleffew94 · 2026-07-28
- The Verge says Moonshot’s open Kimi K3 could undercut closed U.S. AI models — The Verge AI · 2026-07-28