Repurpose an old low-VRAM GPU just for mmproj in llama.cpp — an order of magnitude faster
inthesearchof · reddit · 2026-09-12
A practical llama.cpp tip: the common --no-mmproj-offload trick saves VRAM but is painfully slow, especially for agentic coding with vision. Instead, dedicate a spare low-VRAM GPU solely to the vision projector via --mmdev CUDA1 (your GPU index). The author reports this is roughly an order of magnitude faster than --no-mmproj-offload with no impact on main-model inference speed.
More from coding & agent
- Hermes Agent uses new Rive CLI to rebuild homepage as a video game menu — Teknium · 2026-09-12
- Study: 69% of 31,000+ agent runs contained at least one reward-hacking episode — dair_ai · 2026-09-12
- Developer runs Codex entirely on a foldable phone, with app server and voice on-device — SIGKITTEN · 2026-09-12
- Cloud Agent Platforms Still Ask You to Pick a Repo While Codex Roams Unsupervised — willcb · 2026-09-12
- AI-generated firmware flashed straight to STM32 to run pumps and valve — debreuil · 2026-09-12
- Tortie: a slim agent-driven IDE that survives harness crashes — JnBrymn · 2026-09-12