38 hours on one RTX 3060: SLQ quantization edges out AtomicChat GGUF at same size
maximelabonne · x · 2026-11-02
- Benjamin Marie ran ISTA's Statistically-Lossless Quantization (SLQ) for 38 hours on a single RTX 3060, searching for a better mix of tensors from GGUFs released by AtomicChat and LiquidAI, and published LFM3-1.7B-SLQ-GGUF on Hugging Face.
- Best configuration: 97.74% EAR@10 vs 97.52% for AtomicChat's AD-Q6K at essentially the same size (1.588 GB) — better, though he doubts it's noticeable in real use.
- Ongoing RCO and GSQ refinement pushes checkpoints above 98 EAR; a 1 GB LFM3 1.7B release is planned. He also reports strong evidence that EAR correlates with task accuracy at much lower cost, with details promised next week.
More from Infra
- Blackwell servers now fetch $2.4M each in China, prices tripled in four months — teortaxesTex · 2026-11-02
- Apple's MLX and mlx-lm: Local LLM Inference on Apple Silicon — goyalshaliniuk · 2026-11-02
- llama.cpp: Running GGUF-Quantized LLMs on CPUs, GPUs and Apple Silicon — goyalshaliniuk · 2026-11-02
- 5 Inference Engines Powering Modern AI: vLLM, SGLang, TensorRT-LLM, llama.cpp, MLX — goyalshaliniuk · 2026-11-02
- Particle-1 trails Opus 5.5 by 3.4 points on memory eval but costs 1/23 per correct answer — garrytan · 2026-11-02
- Nvidia's 27-year market cap history: from $800M to $5T, with 70% crashes along the way — mitchdeg · 2026-11-02