Strata DualGPU runs two mismatched NVIDIA GPUs for 5.6x faster local decoding
GabGarrett · x · 2026-10-28
Open-source Strata DualGPU makes two different NVIDIA GPUs work together on a PC with just 32GB RAM. On RTX 5080 + 4060 Ti with Qwen3.8 Flash Next (1.5 IQ2XS), code decode jumps from 29 to 143 tok/s — 5.6x over a single 5080 — and 161 tok/s with a fine-tuned draft layer; 32K prompt processing hits 1,900 tok/s. The trick: RAM-resident experts with pipelined windows across both GPUs instead of leaving one idle. Available on GitHub.
More from Infra
- Statistically Lossless Qwen3.8 27B GGUF Quants Hit ~99% EAR at 23GB, Feels FP8-like — Dany0 · 2026-10-28
- Tonawanda Seneca Nation opposes data center on wetland at People's AI Assembly — AINowInstitute · 2026-10-28
- Kimi K3.1 teased to adopt DSA like DeepSeek V3.2 for better inference economics — zephyr_z9 · 2026-10-28
- VLM Run Gateway Launches: One API for Open-Weight OCR, VLM and Vision Models — spillai · 2026-10-28
- Hyperscalers issue $228B in bonds YTD, nearly 14x the 2024 total — Beth_Kindig · 2026-10-28
- Bittensor subnet Targon burns ~$2.1M of tokens from organic GPU revenue — markjeffrey · 2026-10-28