Paper Benchmarks Black-Box vs White-Box LLM Confidence Estimation Across 11 Open-Weight Models

kalyan_kpl · x · 2026-11-21

A new paper examines how LLMs can perform zero-shot text classification while also estimating prediction confidence—without any fine-tuning—to address the problem of knowing when to trust model outputs.

The takeaway: LLMs already work as confidence-aware zero-shot classifiers out of the box, but calibration quality varies substantially across confidence estimation methods.

Original post →

More from Research

Research channel →