e-means quantization shrinks Qwen3.5-4B to 1.23GB while keeping 91.9% performance
kastnerkyle · x · 2026-11-11
Developer shinboson swapped k-means for the new e-means algorithm in codebook generation within his quantization compiler, shrinking Qwen3.5-4B from 8.41GB to 1.23GB while retaining 91.9% of original performance — roughly SOTA, comparable to bonsai 2. e-means beats k-means with 10x fewer epochs, ships in FAISS, and needs no tuning beyond k. At this size, decode speed is no longer memory-bandwidth-bound, and he's exploring faster larger models on consumer hardware.
More from Infra
- Polymarket prices 28% odds of a state data center moratorium; Torus raises $500M — Polymarket · 2026-11-12
- AWS-wrapper cloud startup shuts down just one month after seed round — thedealdirector · 2026-11-12
- Utah startup Torus raises $500M for shipping container-sized AI data centers deployable in 4 months — Polymarket · 2026-11-12
- Greylock reading group on AI accelerator design: more transistors no longer mean more performance — _shreya_s · 2026-11-12
- Dennard Scaling Is Dead: Automated Chip Design Could Shrink Hardware Cycles — _shreya_s · 2026-11-12
- Two smuggling scandals put Nvidia's China denials to the test — fortune · 2026-11-12