Are 22 GB modded RTX 2080 Ti cards a viable budget local LLM option?
Dave86ch · reddit · 2026-10-16
Reddit user Dave86ch is building a low-budget local inference server for a small pilot running quantized Qwen 9B and is considering modified RTX 2080 Ti cards with 22 GB VRAM (doubled from the stock 11 GB) sourced on Alibaba.
Listed specs: 352-bit bus, 14 Gbps memory, 250 W TDP, 8+8 pin power, blower cooler. The seller provided a stress-test video showing gpuburn at 100% load, 200–245 W, 69–74 °C, with nvidia-smi reporting 22,528 MiB.
Open questions the author raises:
- Long-term reliability of the VRAM modification under LLM workloads
- Compatibility with CUDA, llama.cpp, Ollama, and vLLM
- Whether all 22 GB is actually stable; Turing driver/compute-capability limits
- Whether it's worth the risk vs. paying significantly more for a 24 GB RTX 3090
The author asks whether anyone has actually run LLM inference on these cards and whether it's a reasonable compromise for a pilot rather than production.
More from Infra
- MLCommons launches MLPerf Endpoints, a benchmark that measures inference endpoints the way buyers actually purchase them — TheKanter · 2026-10-16
- Neocloud M&A decoded: CoreWeave, Nebius, Crusoe, Nscale all bet margins vanish beyond GPU scarcity — AccBalanced · 2026-10-16
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Nscale AI datacenter plagued by coolant leaks, transformer fire, 10 safety violations — saranormous · 2026-10-16
- WSL 3.0 ships with native containers and Linux 6.18 kernel, syscall overhead down ~16% — solyarisoftware · 2026-10-16
- Intel's neural-compressor: SOTA low-bit LLM quantization with FP4/NVFP4 support — HaihaoShen · 2026-10-16