Final-Window Pretraining Influences Post-Training Alignment Behavior

A study shows that the specific data used in the final 500 million tokens of pretraining significantly shapes how a model responds to subsequent supervised fine-tuning and preference optimization. Even when models perform similarly on post-SFT benchmarks, their underlying imprints differ.