Running Qwen3.6 27B on Tesla V100: 128K Context Config & Performance
Traditional_Bell8153 · reddit · 2026-08-08
A developer shared their configuration and performance results for running the Qwen3.6 27B model on a Tesla V100 PCIE 32Gb GPU on Reddit.
The setup uses a Q4KM quantized main model paired with a Q80 Multi-Token Prediction (MTP) draft model, tested under a massive 128K context length. The post details the specific llama.cpp parameters used, including Flash Attention, thread counts, and batch sizes, accompanied by a performance screenshot. The author is looking to compare notes with other V100 users.
More from Infra
- Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16
- Meme: driving to the chip factory in 2028 captures AI's compute anxiety — beffjezos · 2026-10-16
- One volunteer maintains the time zone file behind 4 billion phones — danbri · 2026-10-16
- DeepSeek V4.1 Flash on 4x DGX Spark: weights alone are 511GB, the 494 tok/s claim doesn't fit — Aroochacha · 2026-10-16
- Open-sourced Intel GPU Vulkan inference upgrades: 2x ingestion, 60%+ faster inference — Fun-Aardvark-1143 · 2026-10-16
- ELSFP pain points: >3dB insertion loss, costly PMF, harsh packaging requirements — jwt0625 · 2026-10-16