llama.cpp adds --spec-draft-n-ubatch option to optimize VRAM usage
jacek2023 · reddit · 2026-10-14
ggerganov opened PR #30459 on llama.cpp adding a --spec-draft-n-ubatch option that lets users optimize VRAM consumption, useful for tuning speculative decoding in local deployments.
More from Infra
- The Jensen Huang Playbook: How NVIDIA Built CoreWeave as a Buffer Against Its Biggest Customers — AccBalanced · 2026-10-14
- 64GB Nvidia Spark spotted at €6,574 in Europe — €900 more buys the 128GB model — AIFlow_ML · 2026-10-14
- Toby Ord says RL post-training compute neared parity with pretraining; bet on broad compute-capability trends — ben_j_todd · 2026-10-14
- Fractile CEO: smarter AI can guide chip design, but experiments are now the bottleneck — No Priors · 2026-10-14
- China routes surplus solar and wind to AI compute via "East Data, West Computing" — ingliguori · 2026-10-14
- BitRun runs 1-bit LLMs at 127 t/s on M4 Pro CPU, zero compile, after fixing BitNet kernel bugs — The_VibePreneur · 2026-10-14