LLM logprobs as soft classifiers: a year-old idea finally validated
JnBrymn · x · 2026-09-19
The author revisits a blog post he wrote 18 months ago: using LLM single-token predictions (logprobs) as "soft classifiers" that output probabilities for choices.
Key flaw he found then: logprobs often don't match human intuition — ask a model to guess a coin that lands heads 60% of the time, and it returns 100% heads instead of 60%. He believed fixing this would require a training set of statistical, historical and betting events, but never followed through.
Now the idea appears validated (he links a proof), and he jokes about the "I thought of it first" post, wishing he had persisted.
More from Models
- Meta Muse + Jev screens 10,000 candidates to surface top 100 unanswered immunology questions — DeryaTR_ · 2026-09-19
- Braintrust adds Jev as a judge scorer: typed decisions at up to 193.6× speed and 444.6× lower cost — multiply_matrix · 2026-09-19
- GPU price hike hits even the 1080ti, as local LLM token-speed numbers circulate — HankYeomans · 2026-09-19
- I spent $3.40 on Jev in 24 hours: it will be Jev + LLMs, not Jev vs LLMs — gaganghotra_ · 2026-09-19
- GPT-6 Astra builds a flamethrower demo in Three.js from a screen recording — techartist_ · 2026-09-19
- Databricks: shifting just 20% of coding traffic to OSS models dramatically cuts inference spend — Yuchenj_UW · 2026-09-19