Distilling DeepSeek V4 Flash to a 4B model on DGX Spark: 26 hours, 22ms per judgment
Dan_Jeffries1 · x · 2026-09-19
Developer @taroleo shares a local distillation experiment: spending 26 hours on a DGX Spark to distill the judgment capability of the 157GB-weight DeepSeek V4 Flash into a 4B model.
Key takeaways:
- Dropping text/ generation and specializing purely in judgments pushes local inference down to 40ms
- The 4B student, about 1/20 the teacher's size, outperforms the teacher's instant-response mode
- Each judgment takes 22ms, fully on-device
The insight: instead of having a big model emit full JSON, strip generation down to classification-only and gain order-of-magnitude speed. A useful reference for distillation + edge deployment.
Related event: Developer Distills DeepSeek V4 Flash into 4B Model on DGX Spark in 26 Hours(2 posts)→
More from Infra
- Malaysia poised to become a compute superpower with both US and Chinese chips by next year — basedjensen · 2026-11-04
- ByteDance details USDB storage system: 85.8% lower write latency for recommender sequences — _reachsumit · 2026-11-04
- Oracle cancels $7B chip leasing deal with Tencent, FT reports — SumitGup · 2026-11-04
- Inference engine shrunk to 1023 bytes, still runs SmolLM2-135M — cneuralnetwork · 2026-11-04
- Musk says space data centers are 'looking pretty appealing right now' — elonmusk · 2026-11-04
- Data centers are turning to old jet engines as the wait for power grows — IndependentCoast3974 · 2026-11-04