Sable proves bit-exact LLM inference: Qwen2.5-32B verified on one GPU in 92.5s
MoonMath-ai · hf · 2026-11-11
MoonMath.ai open-sourced Sable, a cryptographic argument system proving bit-for-bit that an integer-quantized transformer produced a given token sequence.
Design
- Fixes a floating-point-free integer function FO where rounding, softmax and sampling are fully input-determined
- Constraint system over the 31-bit KoalaBear prime; batches zerochecks and LogUp-GKR lookups into one sumcheck per domain
- Multilinear commitments from Reed-Solomon codes, BLAKE3 Merkle trees and BaseFold; the prover streams layers on a single GPU
Results
- One RTX PRO 6000 Blackwell proves all 64 layers of Qwen2.5-32B in 92.5s (4,096 tokens) and 398s (16,384 tokens); also covers Qwen3-32B and MoE Qwen3-30B-A3B
- Proof costs 41-50x serving time (6.4 cents per 4,096-token request), 40-64x faster per parameter/token than zkLLM
- FO's greedy tokens match BF16 on 86.3% of WikiText-2 positions; within 1.6 pts on MMLU and 1.7 on GSM8K for Qwen3
- CPU-core verifier checks proofs in 19-56s; soundness error 2^-103
Not zero-knowledge, so it targets open-weight models. Code available on GitHub.
More from Infra
- Stratechery: Intel vs AMD's diverging earnings and Intel's need for an old-new path — Stratechery · 2026-11-11
- Alibaba files to list chip-design unit T-Head in Hong Kong to fund AI push — pstAsiatech · 2026-11-11
- AWS CEO expects to be short on compute 'for the next couple of years' as demand outstrips supply — firstadopter · 2026-11-11
- Natural language search over autonomous driving data with Cosmos-Embed1 + OpenSearch — 4310sy · 2026-11-11
- Fully local MiniMax H3 Image + LTX-2.5 pipeline generates a shot in under 5 minutes on one RTX 5090 — Hailuo_AI · 2026-11-11
- Dev claims ~700 tokens/s running Qwen 3 27B on a single RTX 5090 — IgorCarron · 2026-11-11