Recursive Training Causes Five-Fold Variance in Model Collapse
September 11, 2026
A study of 13 model checkpoints undergoing five generations of recursive training reveals a wide fragility spectrum. Unique 4-gram outcomes ranged from 0.187 to 0.940, indicating that while some models resist degradation, others collapse into repetitive fragments regardless of seed or data mixture changes.
HOW THIS AFFECTS YOU
●
builderBe cautious of data loops in your fine-tuning pipelines as specific architectures may be more prone to collapse.
●
researcherYou should evaluate model-specific robustness to synthetic data contamination during pre-training.