DeepSeek V4 Flash Shrunk to 57GB, Writes a Compiler
hacklas · hn · 2026-08-17
A developer compressed DeepSeek V4 Flash 0731 from 284B to 57GB parameters while preserving reasoning and coding capabilities. Tested on an M3 Max, the model wrote an ARM64 C compiler in under an hour, passing Fibonacci and FizzBuzz tests.
Key Techniques:
- Efficient Quantization: Uses mlx-iqk with IQK tensor encoding, outperforming llama.cpp and bare MLX.
- Expert Pruning: Removed 80B parameters less critical for coding using the REAP technique.
- Layout Optimization: Switched to k-contiguous layout for faster Metal performance.
It runs on 32GB Macs (128K context, 5 tok/s) and even on 16GB M1 Airs (1.39 tok/s). Code is available on GitHub.
More from Infra
- First AA-AgentPerf results: NVIDIA Vera Rubin NVL72 delivers up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- Vera Rubin NVL72 first benchmarks: up to 5.7x throughput per MW vs GB300 — ArtificialAnlys · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16