Privileged Self-Practice improves multi-turn agent training over OPSD
September 25, 2026
On-policy self-distillation (OPSD) can degrade agent performance by teaching models to act on unobserved privileged information. Privileged Self-Practice (PSP) solves this by moving privileged data from the loss function to the sampler, using an analyzer model to inject context during failed rollouts within a GRPO objective.
HOW THIS AFFECTS YOU
●
builderThis offers a more stable method for training agents using reinforcement learning with privileged information.
●
researcherYou can avoid performance degradation in multi-turn agents caused by improper self-distillation recipes.