Cactus Hybrid adds a 68k-parameter probe so Gemma 4 can route uncertain answers

Henrie_the_dreamer · reddit · 2026-07-23

Cactus Hybrid gives Gemma 4 a learned wrong-answer detector

Cactus says it post-trained Gemma 4 E2B to estimate when it is likely wrong, then use that confidence to route only a fraction of queries to a larger cloud model.

The system adds a 68k-parameter probe layer that reads an intermediate hidden state and predicts p(wrong).
Confidence is returned as structured data instead of being parsed from generated prose.
On 12 held-out benchmarks across text, vision, and audio, the probe averages 0.814 AUROC, versus 0.549 for token entropy.
The authors say the probe was trained on zero audio data yet still reaches 0.79–0.88 AUROC on several audio benchmarks.
By routing only 15–35% of queries to Gemini 3.1 Flash-Lite, they claim Gemma-4-E2B can match it on most benchmarks.
Weights and integration code are published for Hugging Face, Transformers, MLX, llama.cpp, and more.

The key idea is a compact confidence probe inside the checkpoint, enabling on-device-first execution with cloud fallback when the model looks uncertain.

Original post →

Cactus Hybrid adds a 68k-parameter probe so Gemma 4 can route uncertain answers

More from coding & agent