Correct-Token Retention Regularization for Uniform-State Diffusion Models

Uniform-state diffusion models currently struggle with token retention, frequently revising 173–270 of 512 positions even during greedy-tail decoding. The authors identify a validation NELBO decomposition deficit where training fails to penalize incorrect predictions at clean positions, proposing a new regularization method to stabilize sampling diversity.