68M encoder beats 435M DeBERTa: per-option scoring wins for closed-set tool decisions
Tardelli Ronan Coelho Stekel · hf · 2026-10-27
The Crivo team systematically studies how small an encoder can be for closed-set decisions (e.g., tool selection) at the end of LM pipelines, and whether options should be read jointly or per-option.
Key results:
- At 32M parameters across three seeds, per-option scoring beats joint scoring by 5.8 points of chance-corrected skill on held-out samples of a 37-benchmark decision suite (2.8 at matched compute)
- The advantage requires a reranker initialization whose query-candidate pair input matches per-option reading; from raw encoders the two readers are indistinguishable
- Per-option scoring needs 5–13x more encoder passes: at most 2x median latency, but >4x mean
- Of six released joint encoders (17M–1B), the 68M and 150M models beat a 435M DeBERTa-v3 zero-shot classifier by 4.9 and 5.9 accuracy points on 209 overlap-audited System1-Arena tasks, on par on 145 unseen tasks
- Smallest model answers in 14ms median on 16 threads vs 728ms for the classifier; reordering options flips 12–23% of joint models' answers (zero for per-option)
Weights are open, showing tiny encoders can replace LLM prompting for tool-routing tasks.
More from Research
- EdgeRock architecture keeps KV cache at 72 MiB for 256K context, 99.1% smaller than Qwen3.5-4B — alifcoder · 2026-10-27
- Equivariant Music Transformer accepted to ISMIR 2026 with Best Paper Award nomination — umpedronosapato · 2026-10-27
- Deedy shares links to the OpenAI and Claude ETH-refutation documents — deedydas · 2026-10-27
- Claims: OpenAI and Claude models independently found subexponential 3-SAT algorithms, refuting ETH — deedydas · 2026-10-27
- ETH humanoid uses head-mounted LiDAR for onboard terrain mapping in 30ms — ChongZzZhang · 2026-10-27
- NYU and UT Austin study finds AI models overuse language patterns linked to reading difficulty — andrewaltair · 2026-10-27