Worst-Case Constrained RL for Robust LLM Reasoning Distillation

To prevent reward hacking during distillation, this method formulates reasoning transfer as a constrained reinforcement learning problem. It optimizes for task rewards while enforcing a worst-case constraint on logical violations, ensuring small student models do not sacrifice reasoning integrity for high teacher-agreement scores.