Pivot-SD: Optimizing Masked Diffusion Language Models via High-Impact Pivots

Pivot-SD is an offline self-distillation framework that improves masked diffusion language models by supervising only high-impact tokens, or pivots. It uses an information-gain metric to identify specific token commitments that most significantly reduce uncertainty and shape the model response.