[arXiv]score: 0.24
What Is a Repeated Token Worth? The Scaling Geometry of Multi-Epoch Pretraining
October 6, 2026
Repeated tokens lose value relative to fresh data as a function of extra epochs divided by unique tokens per parameter. While a second epoch remains nearly as valuable as a fresh one, value drops to half after a critical epoch count determined by training budget per parameter rather than model size.
DAILY DIGEST
you don't check 9 sources — we do. one email every morning, read in 2 min. free. unsubscribe anytime. privacy