Running a 22B video model on a 14 GB Vega 7 iGPU: patches, tricks, timings
shirosoro · reddit · 2026-11-01
A developer got diffusion and even video generation running on a ThinkPad T14s with only an AMD Ryzen 4750U (Vega 7 iGPU, 14 GB shared RAM, no matrix cores or bf16) via stable-diffusion.cpp + Vulkan, open-sourcing everything in the vega-igpu-diffusion repo.
Deliverables
- ggml-vega.patch: fused kernels, two-wave matmul tile, tiled f16 weights for GCN/Vega
- sd-cpp.patch: speedups for Anima/Wan VAE/LTX-2.5, on-disk prompt-embedding cache, freeing diffusion weights before VAE decode
- GTK4 GUI (queue/history/img2img/video), build scripts, full measurement log
Timings (25 W limit): Anima 2B turbo 512×768 in 23 s; SDXL+DMD2 832×1216 in 38 s; LTX-2.5 22B distilled (Q2KXL), 17 frames in 3.3 min — a 22B video model on a 14 GB machine.
Key tricks: encode prompt on CPU with --mmap, drop the audio stream (3 GB saved), free weights before VAE decode, never quantize norm weights (Q4K silently broke prompt following), Q3K decoding 4 values per load (825→1068 GFLOPS), casting activations to f16 (6% gain). Pixel-space upscaling beat LTX's latent upscaler for anime.
Linux/Vulkan only, patches not upstreamed; timings are single-machine reference values.
More from Infra
- Sequoia's Shaun Maguire hits back at SpaceX orbital-compute model critics: four core variables matter — shaunmmaguire · 2026-11-01
- Infernix inference engine update targets RTX 5090 with new Qwen3.8 quantized models and lossless mode — Suspicious-Mark-5450 · 2026-11-01
- AI stocks shed 13% in October as Wall Street questions data center payoffs — Saul_Loveman · 2026-11-01
- Local STT shootout: Nemotron 3.5 Transcribe beats Fable on noisy audio — CryptoxPathy · 2026-11-01
- Old PC relabeled as an "AI system" sells for €5,400 after sitting at €750 — sernamenotdefined · 2026-11-01
- Turning AI data centers' waste heat into tea and saunas? A cozy proposal — emax · 2026-11-01