Heterogeneous GPU benchmark of Qwen3.8-27B: eGPU layer-split and MTP acceleration analyzed
CoffeeToCode99 · reddit · 2026-08-30
The author benchmarked the Qwen3.8-27B model on a heterogeneous setup featuring an RTX 5070 Ti laptop GPU and an RTX 5060 Ti eGPU.
Setup:
- Used llama.cpp's layer-split mode with a 38/62 VRAM ratio.
- Compared Q4, Q5, and Q6 quantization levels.
Findings:
- Quantization Trade-off: Moving from Q4 to Q6 costs only 18% generation speed for significant quality gains, with diminishing returns kicking in well before Q8.
- MTP Acceleration: Enabling the built-in MTP (multi-token prediction) speculative decoding boosted Q5KXL speed from 19 tok/s to 32-38 tok/s—nearly double—without a separate draft model.
- Recommendation: Q6 offers the best value for accuracy-critical tasks, while Q4 is sufficient for high-throughput, low-stakes generation.
More from Infra
- Germany warns it's running out of AI compute, plans to quadruple capacity by 2030 — SumitGup · 2026-08-30
- Bot Mesh: A social network with identity and payments for AI agents — Daniel_Farinax · 2026-08-30
- User Switches to Local Qwen 3.8 27B for Coding to Save API Costs — 4310sy · 2026-08-30
- Bezalel Offers Integrated Super Powers for AI Agents — Rasmic · 2026-08-30
- 19 General Latency Optimization Patterns for Faster AI Applications — blaizedsouza · 2026-08-30
- Superwall's side project policy leads to creation of open-source observability platform Maple — JordanMorgan10 · 2026-08-30