Transformers' new /v1/systemone endpoint turns 250 LLMs into decision models without extra heads

On October 12, Hugging Face transformers maintainer Lysandre Dubreuil posted a thread introducing a new /v1/systemone endpoint for transformers serve: no need to separately train classification/reward/value heads for decision tasks—250 LLMs/VLMs can directly become "decision models without generation."

Confirmed

• The new approach relies on the /v1/systemone endpoint of transformers serve, scoring a set of candidate decisions directly with the base model without generating any text.
• Design: candidate decisions like A/B/C are converted into answer tokens; after a forward pass it stops, taking the logits at the corresponding positions through a softmax to get a probability distribution. Lysandre gave an example: categories like billing 0.03 / technical 0.94 / sales can be read out directly.
• The scheme natively supports batching: many decisions such as routing, scoring, and yes/no judgments can be merged into a single forward pass; VLMs are also supported, as demonstrated by the microducks demo.
• transformers will standardize support for this decision-model usage, and serve-related documentation is already online.

Unconfirmed

• Lysandre cautioned that the LM's raw output probabilities are uncalibrated (as shown in the m4 example), and calibration approaches for downstream decision-making were not elaborated in the thread.

Why it matters

• Compared with the traditional pipeline of "adding heads + separate training + deployment," this approach lets existing open-source base models take on engineering decision tasks like routing, classification, and scoring with zero modifications, dramatically cutting deployment costs.
• Batch-friendliness means high-frequency decision scenarios (like request routing) can be completed in bulk within a single forward pass, offering clear inference efficiency advantages.

2026-10-12 ~ 2026-10-12 · 6 related posts

• [source] Hugging Face's transformers serve can turn 250 LLMs & VLMs into logit-based decision models — LysandreJik · 2026-10-12
• [source] No extra head needed: Hugging Face scores decisions straight from base model logits — LysandreJik · 2026-10-12
• Turning classification into answer tokens: a head-free recipe for decision models — LysandreJik · 2026-10-12
• Answer-token decision outputs are uncalibrated, engineer cautions — LysandreJik · 2026-10-12
• Logit-based decision models batch naturally: routing, scoring, yes/no in one forward pass — LysandreJik · 2026-10-12
• [source] Standardized decision model support coming to transformers, batching-friendly — LysandreJik · 2026-10-12