Flux-OPD Stabilizes On-Policy Distillation with Evolving Contexts

Flux-OPD addresses instability in distilling LLMs within open-ended domains by decomposing the reverse KL objective to manage teacher-context conflicts. The method uses evolving contexts as in-training supervision, effectively steering the student toward the geometric mean of context-conditioned teachers.