I-SDPO: Instance-Level Adaptive Self-Distillation Policy Optimization

I-SDPO mitigates signal loss in Group Relative Policy Optimization (GRPO) when all sampled responses in a rollout group are incorrect. The method applies instance-level routing to switch between GRPO and privileged self-distillation, using dense token supervision only for unsuccessful groups to prevent biased teacher imitation from hindering reward-improving updates.