Running a 763B-parameter model in ~305GB RAM near losslessly, thanks to DeepSeek's paper
gnukeith · x · 2026-10-29
Developer gnukeith highlights DeepSeek's quantization work: a 763B-parameter model can run near-losslessly in about 305GB of RAM. He argues this points to AI becoming significantly cheaper and more capable over time rather than more expensive.
Related event: Developer hails DeepSeek efficiency: KV Cache shrinks 437x in three years(3 posts)→
More from Infra
- Qwen3.8 Flash Next at IQ3_S hits 275 tok/s on a single RTX 5080 via Basalt fork — yehyakar · 2026-10-29
- Google Cloud CEO: Apple Scaling Foundation Model Pretraining on Latest TPUs — shayan_ · 2026-10-29
- FlashAttention vs standard attention in one chart: the bottleneck is data movement, not compute — cneuralnetwork · 2026-10-29
- Particle-1 inference goes live on OpenRouter, claiming 28-52% fewer output tokens — ycombinator · 2026-10-29
- Qwen3.8-27B EXL3 quant hits lowest KLD of 0.0002 in just 23 GB — suprjami · 2026-10-29
- MLX-Serve 26.11.1 rewrite: plugin architecture adds models and quants without recompiling — TheMoonMidas · 2026-10-29