MixSD Reduces SFT Forgetting by Mixing Expert and Naive Model Conditionals

Fine-tuning on human or external targets causes forgetting because low-probability token sequences pull models away from their pretrained distribution. MixSD constructs supervision by blending two conditionals of the base model itself — one with the injected fact in context, one without — keeping targets distribution-aligned without any external teacher.