Running Qwen3.6 Quantized on RTX 5080: Performance Trade-offs
boyeardi · reddit · 2026-07-07
一位用户在 RTX 5080 + 32GB DDR4 上本地运行了 Qwen3.6：27B 在 Q3KM 下约 60 tok/s，35B 在 IQ2 下约 190 tok/s，但精度从 Q6 下调带来的准确性损失明显。该用户另有一张闲置的 1080ti（11GB），考虑用其做 tensor split 分担负载，并询问 16GB 显存用户的实际方案与上限。
More from Infra
- Engineer with thousands of fabricated devices pours cold water on photonic crystal modulator yields — jwt0625 · 2026-10-26
- Anthropic S-1 reveals 48 cents of compute cost per dollar of Q2 revenue — nmasc_ · 2026-10-26
- Persistent AI joins UR network to route private AI inference with encrypted, IP-hidden traffic — 0xSammy · 2026-10-26
- Anthropic's disclosed compute commitments: $114.5B Google, $161.2B Broadcom top the list — BenBajarin · 2026-10-26
- SpaceX strikes $40B deal with Apollo Global to lease AI chips and data center hardware — Polymarket · 2026-10-26
- SpaceX AI revenue up 830%, Claude designs 318K antivenom proteins in 64 hours — PeterDiamandis · 2026-10-26