Letting a 65B model quantize itself: Ripple squeezed to 30.3GB for local serving

Azaliamirh · x · 2026-11-17

An experiment from the RicursiveAI hackathon: the author argues that running coding agents locally hinges on serving models locally, and that quantization is fundamentally a search problem — choosing which layers to quantize and which to keep. He let Ripple 65B act as an agent to search its own quantization scheme, producing a 30.3GB version that holds up well enough for local use.

Related event: Ripple-65B Hackathon: Self-Quantization Shrinks 60GB Model to 30GB(2 posts)→

Original post →

More from coding & agent

coding & agent channel →