Divergence Between LLM Linguistic and Internal Confidence
August 31, 2026
An evaluation of 30 models across 10 tasks shows that linguistic confidence often fails to track internal logit-based confidence or semantic entropy. Instruction-tuned models frequently report higher confidence but exhibit worse calibration and larger confidence gaps than base models.
HOW THIS AFFECTS YOU
●
builderYou cannot rely on a model's verbalized confidence to judge the reliability of its outputs in production.
●
researcherYou should be cautious when using self-reported confidence as a proxy for model uncertainty in research.