Sampling More, Getting Less: Calibration is the Diversity Bottleneck in LLMs

Researchers introduce a validity-diversity framework attributing LLM output collapse to probability mass allocation across valid/invalid continuations during decoding, identifying calibration as the diversity bottleneck in language model inference.