Config: Running Qwen3.8 27B Q8 on 3x RTX A4000
LevelSoft1165 · reddit · 2026-08-21
A user shared a config for running the Qwen3.8 27B Q8 quantized model on 3x RTX A4000 (16GB) GPUs using llama.cpp. With MTP optimization enabled, the setup achieves an average speed of about 23 tokens/s. The user is seeking advice for further performance improvements.
More from Infra
- Haight Street Compute Market: a one-day trading floor for GPU deals in SF — ctjlewis · 2026-10-16
- Temporal Serverless Workers on Cloud Run: queue-driven autoscaling, scale to zero — rseroter · 2026-10-16
- LMA brings day-0 model support to TPU for the first time, spanning NVIDIA, AMD, Intel via SGLang — ying11231 · 2026-10-16
- Semtech's analyst-day slide cleanly explains the four signal-integrity impairments — BenBajarin · 2026-10-16
- Fortytwo Engine boosts Qwen3.8-27B accuracy 7% and cuts tokens 25% via Swarm Inference — rohanpaul_ai · 2026-10-16
- Berkeley scholars: concentrated chip supply chain makes a hardwired frontier AI pause feasible — wfithian · 2026-10-16