DIY-Jev: no fine-tuning needed — reading true/false logits hits 75.5% on 32k benchmark
Malfeitor1235 · reddit · 2026-09-21
Developer VakeDomen released DIY-Jev, a Jev-style inference setup that needs no NLI fine-tuning or classifier head and works with completely unmodified open-weight LLMs.
Method: each candidate answer becomes a boolean verification — after building a <state>/<question>/<options>/<candidate> prompt, instead of generating text the code reads the true/false logits, subtracts them per candidate, and softmaxes the differences. The expensive prefix is evaluated once; candidate branches (differing only in the last tokens) are batched through llama.cpp.
Results on a 32,235-example benchmark (laptop RTX 5090 24GB):
- Qwen3-4B: 65.0% at 27 req/s
- Qwen3-27B: 75.3% at 2.9 req/s
- Qwen3.6-35B-A3B: 75.5% at 5.3 req/s
Notably, the same model with zero training outperforms the OpenJev fine-tune. The author cautions it's not a strict Jev reproduction and the benchmark isn't perfectly apples-to-apples. Code: GitHub DIY-Jev.
More from Research
- Researchers formally verify the Kubernetes control plane with a compositional CORE spec — tianyin_xu · 2026-09-21
- Astra shows any 3D/4D prior can be distilled into VLMs, a new embodied AI paradigm — mariyaivasileva · 2026-09-21
- ReCouPLe: Reason-Augmented Preference Learning Boosts Reward Accuracy 1.5x Under Shift — burny_tech · 2026-09-21
- kalomaze rethinks 'cribbing SSL backbone features' after peers hit multimodal walls — kalomaze · 2026-09-21
- Closed-loop spatial understanding from just two wrist cameras, no gripper feedback — ChongZzZhang · 2026-09-21
- DeepMind's weather model ingests raw satellite radiances hourly, ending the 6-12h analysis lag — burny_tech · 2026-09-21