Cactus Hybrid adds a 68k-parameter probe so Gemma 4 can route uncertain answers
Henrie_the_dreamer · reddit · 2026-07-23
Cactus Hybrid gives Gemma 4 a learned wrong-answer detector
Cactus says it post-trained Gemma 4 E2B to estimate when it is likely wrong, then use that confidence to route only a fraction of queries to a larger cloud model.
- The system adds a 68k-parameter probe layer that reads an intermediate hidden state and predicts p(wrong).
- Confidence is returned as structured data instead of being parsed from generated prose.
- On 12 held-out benchmarks across text, vision, and audio, the probe averages 0.814 AUROC, versus 0.549 for token entropy.
- The authors say the probe was trained on zero audio data yet still reaches 0.79–0.88 AUROC on several audio benchmarks.
- By routing only 15–35% of queries to Gemini 3.1 Flash-Lite, they claim Gemma-4-E2B can match it on most benchmarks.
- Weights and integration code are published for Hugging Face, Transformers, MLX, llama.cpp, and more.
The key idea is a compact confidence probe inside the checkpoint, enabling on-device-first execution with cloud fallback when the model looks uncertain.
More from coding & agent
- BELL: Open-Source Stack Combines Browserbase, Eve, iMessage and Stripe Link for Personal Agents — jeff_weinstein · 2026-10-27
- Legora runs 2,000 parallel agent evals on Modal Sandboxes for legal AI — blelbach · 2026-10-27
- Two engineers rebuilt a FedEx supplier's billing core in 3.5 months vs an estimated 7-8 — alex_verem · 2026-10-27
- GPT-6.1 Terra gains ~20 points on Terminal-Bench 4.0, rivalling GPT-6.1 Sol — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27
- Shift-1 router cuts coding agent spend up to 55% while keeping SWE-bench accuracy — ypatil125 · 2026-10-27