LEV-2.6B jumps 63.2% to 76.2% on JevBench as depth goes 1→4; on-demand depth saves little

RobertTLange · x · 2026-11-08

RobertTLange shares empirical results from the ThinkJev/LEV line of work on adaptive reasoning depth:

Confidence thresholds don't beat fixed depth mixes: depth-1 answers are 94.8% accurate on calibration data but only 78.9% on new sources, at nearly identical confidence — logit-based confidence is unreliable for routing.
Depth does pay: LEV-2.6B improves from 63.2% to 76.2% on JevBench going from depth 1 to 4 (the table shows 200ms latency at depth 4, with LEV models beating equal-data dense controls like Qwen3 baselines on average).
On-demand depth saves 15–24% of compute for under 1 point of accuracy loss, mostly by stopping early on easy items (1.2/2.4/3.0 steps on easy/standard/hard).
But the gain over simply mixing fixed depths is marginal: only +0.1–1.0 points.

More from Research