Decoupling of Internal and Verbalized Probabilities in Large Language Models

Research shows that an LLM's internal sampling distribution and its verbalized confidence are not inherently aligned. The degree of alignment depends on whether training data frequencies match explicit probabilistic assertions, limiting the reliability of using verbalized uncertainty as a proxy for model distribution.