DeepSeek V4 Flash Shrunk to 57GB, Writes a Compiler
hacklas · hn · 2026-08-17
A developer compressed DeepSeek V4 Flash 0731 from 284B to 57GB parameters while preserving reasoning and coding capabilities. Tested on an M3 Max, the model wrote an ARM64 C compiler in under an hour, passing Fibonacci and FizzBuzz tests.
Key Techniques:
- Efficient Quantization: Uses mlx-iqk with IQK tensor encoding, outperforming llama.cpp and bare MLX.
- Expert Pruning: Removed 80B parameters less critical for coding using the REAP technique.
- Layout Optimization: Switched to k-contiguous layout for faster Metal performance.
It runs on 32GB Macs (128K context, 5 tok/s) and even on 16GB M1 Airs (1.39 tok/s). Code is available on GitHub.
More from Infra
- Llama-Mobile: 2.7-Bit Quantization Shrinks Llama 3.2 Vision 11B to 3.7GB for Phones — Luka Ribar · 2026-08-24
- DSCO Router Launches Unified Gateway for Multi-Model Routing with BYOK Support — arthurcolle · 2026-08-24
- Open Source RobotSoul: Persistent Identity for Agents After Context Resets — robauto-dot-ai · 2026-08-24
- Offloading MoE models to RAM causes slow prefill speeds — former_farmer · 2026-08-24
- Etched Raises $1B Led by Jane Street to Validate Architecture-Agnostic AI Chips — TheTuringPost · 2026-08-24
- ConvRot Quant joins llama-cpp: Q6 accuracy nears Q8 quality — giveen · 2026-08-24