One token is all you need: hacking LLMs into classifiers with logit_bias and max_tokens=1
AAAzzam · x · 2026-09-21
The author recalls the pre-Jev days of slinging logitbias + maxtokens=1 to hack LLMs into classifiers: via constrained sampling you can micromanage ChatGPT's output down to a single token, but that one token can do a lot — act as a classifier, a logic gate, or choose tools deductively. Material comes from his talk at Chroma this week. A great example of a non-obvious technique: constrain candidate tokens with logitbias for fully deterministic, parseable outputs, well suited to routing and agent decision points.
More from coding & agent
- PM builds a real SaaS in 24 hours with AI agents, sharing his full zero-cost stack — PawelHuryn · 2026-09-21
- Prompt cache restore works, but reusing KDA/Mamba states hits illegal memory access — TheZachMueller · 2026-09-21
- Why this dev hasn't shipped his agent: heavy tokens, slow, CPU-hungry — BLUECOW009 · 2026-09-21
- 128k context with compaction beats raw 256k/1M for quantized models — Informal-Trouble2183 · 2026-09-21
- If your LLM's tool-use trace shows it browsing Bluesky, shut that down immediately — venturetwins · 2026-09-21
- The real bottleneck is memory and compression, not context: dev argues compaction is a crutch — JoelMahon · 2026-09-21