LiquidAI LFM2.5-2.6B Quantization Report: Runs on Raspberry Pi
crusaderky · reddit · 2026-08-07
LiquidAI recently released LFM2.5-2.6B, a new tiny model with benchmarks rivaling much larger ones. A developer conducted detailed tests on various combinations of GGUF model quants and KV cache quants.
Key Findings:
- The model fits on an 8GB Raspberry Pi with no material degradation and on a 4GB Pi with contained degradation.
- DO NOT use the Q4KM quantization.
- For this model, model quant quality degrades faster than KV cache quant.
- Abliteration incurs a flat cost of 0.075 KLD.
- Logarithmic KLD and Top-1% plots lie by showing smooth quality degradation when it's actually a cliff.
More from Infra
- Speculative Decoding Will Reshape LLM API Training Terms of Service — charles_irl · 2026-08-07
- Mixed 180 Consumer GPUs Train 50B Tokens: 100 Interruptions Add Only 2% Cost — bittingthembits · 2026-08-07
- Ex-Meta Researcher: Google's Internal Infrastructure is World-Class, Built for Scale — finbarrtimbers · 2026-08-07
- Cloudflare Unifies Workers AI and AI Gateway into a Single Control Plane — michellechen · 2026-08-07
- Running MiniMax on B200 GPU: 10s Video in Under 2 Minutes — Foreforks · 2026-08-07
- Turso Rewrites Postgres in Rust to Build the LLVM of Databases — JeremyCMorgan · 2026-08-07