Pinocchio: open-source 0.8B model adds calibrated confidence to black-box LLM APIs
A UMD team led by Kevin David Hayes released Pinocchio, an open-source 0.8B model that assigns calibrated confidence scores to outputs of black-box API LLMs like GPT, achieving 0.862 AUROC—addressing the lack of reliable uncertainty estimates in frontier models.
2026-10-02 ~ 2026-10-02 · 3 related posts
- Pinocchio: a lightweight model that adds calibrated confidence to frontier LLM outputs — micahgoldblum · 2026-10-02
- Micah Goldblum's team releases new paper with open models, code and website — micahgoldblum · 2026-10-02
- Pinocchio: an external calibrator brings fast uncertainty estimates to black-box LLM APIs — micahgoldblum · 2026-10-02