Lightning OPD 2.0 Reduces Style Bias in Cross-Teacher Distillation
July 31, 2026
Lightning OPD 2.0 introduces cross-fitted style residualization to mitigate performance drops in on-policy distillation when the teacher model and SFT reference model differ. The method uses rollout-level cross-fitting to isolate reasoning improvements from superficial differences in wording and formatting.
HOW THIS AFFECTS YOU
●
builderYou can improve distillation quality when using mixed-source datasets for fine-tuning.
●
researcherThis provides a method to handle inconsistent teacher-student data provenance.