On-Policy Delta Distillation Improves Multilingual Math Reasoning
August 7, 2026
On-Policy Delta Distillation (OPD^2) utilizes the probability gap between a teacher and base model to improve math reasoning in multilingual settings. Testing on Qwen3 shows significant gains in Korean and Japanese, helping close the performance gap with English.
HOW THIS AFFECTS YOU
●
researcherYou can use probability gaps as a learning signal to improve multilingual model performance.