Idea: replicate JEV-style probability estimates via softmax of LLM logprobs
ngxson · x · 2026-09-18
ngxson proposes replicating jev: any LLM can output probabilities for a context+question+option combination. Frame the answer as binary (yes/no, true/false), then take the softmax of the two logprobs. He hasn't tested it yet and invites others to try.
More from Research
- Epoch AI audits 15 AI benchmarks, finds 9 flawed — 46% of sampled HLE questions broken — morqon · 2026-09-18
- Jack Clark defends OpenAI's RSI measurement research as the '26%' figure spreads — jackclarkSF · 2026-09-18
- Salesforce releases xLAM-2: large action models extended to multi-turn agentic interactions, fully open-sourced — huan__wang · 2026-09-18
- Nemo corpus NER case study: all models misclassify an initiative as an ORG with 90+ confidence — HankYeomans · 2026-09-18
- STOC 2027 CfP out: mandatory public posting and video policies amid generative AI — KonstDaskalakis · 2026-09-18
- Jev and System One Models: fast decision-only AI that outputs typed answers, not text — blaizedsouza · 2026-09-18