Letting a 65B model quantize itself: Ripple squeezed to 30.3GB for local serving
Azaliamirh · x · 2026-11-17
An experiment from the RicursiveAI hackathon: the author argues that running coding agents locally hinges on serving models locally, and that quantization is fundamentally a search problem — choosing which layers to quantize and which to keep. He let Ripple 65B act as an agent to search its own quantization scheme, producing a 30.3GB version that holds up well enough for local use.
Related event: Ripple-65B Hackathon: Self-Quantization Shrinks 60GB Model to 30GB(2 posts)→
More from coding & agent
- LM Studio launches Agent Remote Control for running agents on any device — MatthewBerman · 2026-11-17
- Qwen 3.8 27B at 300-500 tok/s on a 5090 laptop, tuned for parallel agents — tebjan · 2026-11-17
- Ripple-65B Hackathon: Self-Quantization Shrinks Model from 60GB to 30GB with Little Quality Loss — Azaliamirh · 2026-11-17
- Treating agent orchestration like a factory: the 7 jobs of an orchestration layer — blaizedsouza · 2026-11-17
- UCLA and AWS paper: AI agents find math shortcuts but thinking time often cancels the gains — rohanpaul_ai · 2026-11-17
- Pixel art + Gaussian splats game experiments: all code, no assets, but roadblocks ahead — algo_diver · 2026-11-17