4070 Ti Benchmarks: Running Kimi K3, DeepSeek V4, and Qwen 122B Locally
JayB_Official · reddit · 2026-08-25
Using a custom runtime called CRANE V2, the author pushed the limits of running massive MoE models on a consumer Windows PC (Ryzen 7800X3D + RTX 4070 Ti 12GB + 32GB RAM).
Targets: Kimi K3 (2.779T), DeepSeek V4 Flash, Qwen3.5-122B.
Key Findings (Speed vs. Quality):
The author distinguishes between "speed-wall" (max speed) and "quality-preserving" experiments. High speeds often come at the cost of broken logic or garbage output.
- Kimi K3: Canonical control 0.014 tok/s. An altered profile hit 9.3 tok/s but destroyed language quality.
- DeepSeek V4 Flash Q4: Hit 42.26 tok/s with altered static routing, but output was unusable.
- DeepSeek V4 Flash Q3: Via dual-SSD striping, pinned staging, pipelining, and LFU caching, useful throughput improved from 1.14 → 8.08 tok/s while answering queries correctly.
- Qwen3.5-122B: Speed-wall hit 57.38 tok/s (static route, skipped shared expert) producing garbage. Canonical control was 1.89 tok/s.
Technical Optimizations:
- Storage: Treats dual SSDs as one logical store with striped expert reads.
- Caching: Bounded GPU expert caches; found Decayed LFU superior to LRU.
- Pipelining: Overlapped storage transfers and GPU compute.
Conclusion: None of these massive models are practically approved for local use on this hardware yet, but the experiments show how runtime design can extract performance from limited specs.
More from Infra
- Strix Halo + dGPU real-world test: low-context benchmarks oversell the speedup — Hrethric · 2026-08-25
- Debunking the "zombie facts" haunting the data center debate — AndyMasley · 2026-08-25
- Fal releases post-trained H3 model co-optimized with custom inference stack — isidentical · 2026-08-25
- A 4060Ti 16GB running Qwen 27B at IQ3_XXS merged its first full feature branch — o0genesis0o · 2026-08-25
- Optimizing Qwen3-ASR Latency to 70ms to Beat Deepgram — Comprehensive_Quit67 · 2026-08-25
- Agent Memory System Refuses Hallucinations with Provable Deletion and Auditability — External-Fee-8920 · 2026-08-25