Research indicates semantic convergence is inherent in base models rather than being introduced by alignment. SFT acts as a catalyst that reveals and amplifies existing homogeneity, with instruction-tuning stages showing immediate output collapse compared to pre-aligned states.
HOW THIS AFFECTS YOU
●
researcherYou should investigate pretraining data diversity to prevent convergence before alignment begins.