Inducing Process Supervision from Outcome-Only Reinforcement Learning

TIPS (Thinking-Induced Process Supervision) trains generative process reward models using only outcome-based reinforcement learning. By optimizing step-level labels alongside a chain-of-thought based on a final outcome reward, the framework enables process supervision without the high cost of manual step-by-step human annotations.