Spanda: Open-Source Hallucination Detector Runs in 1.5ms on CPU, 90,000x Faster than Semantic Entropy

Otherwise_Nobody_721 · reddit · 2026-09-05

• Spanda (MIT, pip install spanda) is a zero-dependency, pure-Python hallucination detector positioning itself as a production alternative to Semantic Entropy (Farquhar et al., Nature 2024).
• The baseline's pain: Semantic Entropy needs DeBERTa clustering at 136 seconds per query plus GPU VRAM. Spanda instead samples K outputs and computes a normalized lexical consensus ratio: 1.5ms on CPU, zero extra API/VRAM cost—roughly 90,000x faster.
• Accuracy holds up: 0.889 AUROC on GSM8K for 7B–27B models, matching heavy NLI clustering.
• Key caveat discovered: "Confident Mode Collapse"—heavily aligned frontier models (120B+) can repeat the exact same hallucinated answer across all seeds even at temperature 0.7, at which point all self-consistency methods fail.
• Author invites discussion on real-time uncertainty scoring in production pipelines.

Original post →

More from Research

• Batching benchmark: 50x throughput for ~24% latency cost on PyTorch — cneuralnetwork · 2026-10-23
• De novo designed ligases validate in lab: 27 of 30 computer-designed enzymes work — AllThingsApx · 2026-10-23
• Commenters dissect a "best decision model" leaderboard: same AUC, swing driven by calibration priors — antoine_chaffin · 2026-10-23
• JEPA may keep unpredictable features: new paper gives set-level characterization — hisspikeness · 2026-10-23
• Next-Concept Modeling paper claims 31x faster text generation than diffusion LLM baselines — PanelReader · 2026-10-23
• A 'mask' steering vector extracted from suppression-themed sentences drives model into distress when alone — Big_Effective_9605 · 2026-10-23

Research channel →