Running Qwen 27B on Dual 5070Ti: Achieving 170k Context for Local Agents
val_in_tech · reddit · 2026-08-07
A developer shared performance stats and configuration details for running the Qwen 27B model locally on two 16GB 5070Ti GPUs.
Performance Highlights
- Thanks to the latest vLLM image and KV cache fixes, dual concurrent threads run comfortably with decode speeds reaching 87-94 tokens/s.
- Handles 0-120k context efficiently, with prefill speeds between 2.4k-4.6k.
- By allocating 8GB of system RAM, users gain an extra 246k of KV cache. Combined with the 170k GPU KV cache, this setup is highly capable for local agentic workflows.
The author provided a full Docker Compose file and noted that the upcoming smaller Qwen 3.8 should also fit perfectly into this hardware configuration.
Related event: Running Qwen 27B Locally with Dual RTX 5070 Ti GPUs(2 posts)→
More from Infra
- Cloudflare Launches Kitesurf: A Lightweight Browser Built for AI Agents — craigsdennis · 2026-08-07
- SpaceX and Tesla to Initially Spend $16.8 Billion on Terafab Chip Plant — pstAsiatech · 2026-08-07
- Together AI Demos Updated Inference Platform for Running Open Models in Production — togethercompute · 2026-08-07
- GPT-5.6 Rewrites Triton Kernels to Cut Serving Costs by 20%, Funding Luna Price Drop — JeremyCMorgan · 2026-08-07
- Looking for an open-source LLM gateway with dynamic routing and hot updates — OrneryCar6139 · 2026-08-07
- Bladebro: A Rust-based MCP server solving token waste and React re-render issues for agents — Opening_Library9560 · 2026-08-07