Cross-model census analyzes neural text repetition
September 25, 2026
An observational study of 17 pretrained models examines the fixed-point structure of argmax maps to investigate the causes of text degeneration. The findings suggest that repetition behavior is not determined solely by training data but varies across different model architectures like Pythia and Mamba.
HOW THIS AFFECTS YOU
●
researcherThis provides a new framework for comparing how different architectures handle the inherent risk of repetitive text generation.