OPD^2 Distillation Enhances Multilingual Math Reasoning in Qwen3
August 5, 2026
On-Policy Delta Distillation (OPD^2) improves multilingual mathematical reasoning by using the probability gap between a teacher and base model as a learning signal. Testing on Qwen3 shows significant performance gains in Korean and Japanese compared to standard on-policy distillation.
HOW THIS AFFECTS YOU
●
builderThis provides a path to better multilingual math performance without full reinforcement learning.
●
researcherYou can use probability gaps as a more effective distillation signal for multilingual tasks.