antirez's DwarfStar: adaptive VRAM/RAM expert placement runs DeepSeek V4 at 45 t/s
antirez · x · 2026-08-17
antirez (Salvatore Sanfilippo, creator of Redis) shares progress on his DwarfStar inference engine for DGX Station: running DeepSeek v4 PRO at Q2 quantization, routed experts are split between VRAM and RAM, with the engine adaptively migrating experts based on past token-generation usage so the hot set stays in VRAM. It currently reaches 45 t/s and can go faster. He calls this just the start of what's possible, reiterating his claim that DwarfStar could be the Station inference engine.
Related event: Redis Author Optimizes DeepSeek V4 to 45 t/s(3 posts)→
More from Infra
- PufferLib prerelease supports >20M steps/second reinforcement learning — jsuarez · 2026-08-17
- Data center boom drives up emissions as costs outweigh benefits — robleclerc · 2026-08-17
- DeepSeek v4 PRO Q2 hits 45 t/s with VRAM/RAM split on DGX Station — antirez · 2026-08-17
- Model routing should be optimized at the harness layer, not the gateway — agihouse_org · 2026-08-17
- Qwen3.8-27B Benchmarks: RPC inference tests across AMD and Nvidia GPUs — tabletuser_blogspot · 2026-08-17
- AI Spending Gap 625x: Top 1% Spend $7,500/Employee/Month vs Median $12 — rohanpaul_ai · 2026-08-17