Run 262K Context Local Inference on an RTX 3060
UsedMorning9886 · reddit · 2026-07-16
This post shares a local configuration for running Qwen3.6-35B-A3B on an RTX 3060 12GB + 32GB DDR5 setup, aiming to achieve high throughput and full long-context support on limited hardware.
Key optimizations include:
- Forcing MoE expert FFN weights into CPU/system memory
- Keeping attention and KV cache on the GPU
- Disabling speculative decoding and enabling flash attention
- Achieving 262K context and roughly 30 t/s with these settings
The post also mentions combining this local inference engine with open-source local memory solutions for local automation or continuous tasks, avoiding the extra overhead of external retrieval and Docker/cloud infrastructure.
More from coding & agent
- A Forward Deployed Engineer job really has three stages: audit, evals, deploy — blaizedsouza · 2026-07-22
- 438 sealed tests show coding agents prefer DIY over third-party databases — cramforce · 2026-07-22
- Tenable and AWS launch a Black Hat build event for open-source security agents and MCP servers — Dave_Maynor · 2026-07-22
- Codex helps build Valdiluce, an open-world game with climbing, gliding and gondolas — Dimillian · 2026-07-22
- HeyGen adds a media-sourcing skill for coding agents with 75k images and 10k tracks — HeyGen · 2026-07-22
- Agent search bottlenecks are now about variance, not raw latency — rohanpaul_ai · 2026-07-22