Boole-32B-A2.5B-preview: 9GB 1.8-bit MoE hits 565 token/s on a single RTX 5090
d77chong · reddit · 2026-11-13
Recursive released Boole-32B-A2.5B-Preview, an early 1.8-bit quantized version of its flint model, on r/LocalLLaMA.
- Mixed-precision quantization, expert pruning, and custom kernels make it 9x smaller than BF16 and 2x faster than INT4
- Just 9GB, delivering 565 token/s on a single RTX 5090
- Follows their earlier 1-bit Boole-1.7B, responding to community demand for larger and faster MoE models
Weights (GGUF) are on Hugging Face, with a demo repo and a custom llama.cpp fork available.
More from Infra
- SMIC hits record $3.19B Q3 revenue at 97.3% utilization, but capex lags plan — zephyr_z9 · 2026-11-13
- General Reasoning: $750K to prove out the West's first open multi-agent RL recipe for LLMs — rosstaylor90 · 2026-11-13
- Unsloth UD_IQ1_M Quant Saves 6GB but Breaks Tool Calls in Real Use — SomeoneInHisHouse · 2026-11-13
- One DGX Spark runs ripple-65b preview at 10.8 tok/s with coherent long reasoning — DZhang50 · 2026-11-13
- Qwen3.8-Flash-Next: 180B total / 6B active, runs on consumer GPUs via Strata offloading — drdanielbender · 2026-11-13
- Plotly's DataAnalyticsBench: three models that fit in 64GB score a B on 43 analytics tasks — DZhang50 · 2026-11-13