antirez runs DeepSeek v4.1 Flash locally on Macs: 15 t/s single, 25 t/s dual

antirez, author of Redis, systematically demonstrated running DeepSeek v4.1 Flash on consumer Apple hardware: a single 128GB M5 Max achieves roughly 15 tokens/s generation and 800 tokens/s prefill via SSD expert streaming, while two MacBooks linked with RDMA tensor parallelism (50% each) reach 25 tokens/s. He also detailed DwarfStar's three-tier prefill scheduling and the benefits of the shared KV architecture. Combined with Julian Harris's testing showing local coding has reached last year's cloud frontier level, local inference is approaching the cloud experience at zero cost.

Confirmed

• antirez ran DeepSeek v4.1 Flash (Q2 quantization) on a single 128GB M5 Max; the loader streams the experts cache from SSD in just 8 seconds, up to a fully resident encoder (50%) block
• Single-machine generation is about 15 tokens/s with prefill at 800 tokens/s; antirez attributes the speedup to recent "keep the right experts" SSD streaming optimizations
• Two MacBooks with RDMA tensor parallelism (50% each) reached 25 tokens/s generation
• The DwarfStar main branch officially supports DeepSeek v4.1 Flash, and antirez published an English YouTube video demonstrating both SSD streaming and dual-machine setups
• antirez detailed DwarfStar's three-tier prefill strategy: small prefills use the single-token path reusing the experts cache (like decoding), while larger prefills use hierarchical loading
• Fully resident decoders evict the experts cache, but the new SSD streaming implementation is good enough that load latency of a few seconds can be hidden
• DeepSeek v4.1 Flash's shared encoder/decoder KV cache not only saves prefill computation but also enables very fast large prefills in low-memory local setups
• Julian Harris published long-form testing claiming that since August 2026, local AI coding matches "last year's cloud frontier models," running an Opus-level coding agent locally at nearly 2x Claude Opus speed and zero cost

Why it matters

• SSD expert streaming makes models larger than RAM practical on single consumer devices at acceptable speeds; second-scale latency hiding and tiered prefill scheduling show mature engineering
• Shared KV cache architecture further lowers the barrier for local prefill and low-memory inference
• RDMA dual-machine tensor parallelism offers a low-cost scaling path without GPU clusters
• Multiple independent reports point to the same trend: local inference is approaching cloud frontier models at zero cost

2026-09-11 ~ 2026-09-12 · 12 related posts

• antirez runs DeepSeek v4.1 Flash locally on a 128GB M5 Max, SSD streaming surprisingly fast — antirez · 2026-09-11
• antirez hits 25 tok/s running DeepSeek v4.1 Flash across two M5 Max MacBooks via RDMA — antirez · 2026-09-11
• [source] antirez hits 25 t/s running DeepSeek v4.1 Flash across two MacBooks via RDMA — antirez · 2026-09-11
• DeepSeek 4.1 Flash runs mostly off SSD: antirez demos fast inference on a 128GB M5 Max — PMinervini · 2026-09-11
• Running an Opus-level coding agent locally at nearly 2x Claude Opus speed, for free — julianharris · 2026-09-11
• [source] antirez runs DeepSeek v4.1 Flash on a single M5 Max 128GB with 800 t/s prefill — antirez · 2026-09-11
• antirez breaks down DwarfStart's three-tier prefill strategy on local hardware — antirez · 2026-09-11
• antirez explains tiered prefill loading: SSD streaming hides expert cache thrashing in seconds — antirez · 2026-09-11
• antirez: DeepSeek's shared KV cache architecture enables fast big prefills on low-memory local setups — antirez · 2026-09-11
• antirez: DeepSeek v4.1 Flash Shared-KV Architecture Enables Fast Local Prefills — antirez · 2026-09-11
• [source] antirez adds DeepSeek v4.1 Flash to DwarfStar, tests it on dual M5 Max MacBooks — antirez · 2026-09-12
• Antirez Tests DwarfStar: DeepSeek v4.1 Flash in 2-bit Quantization — Short_Regular_7191 · 2026-09-12