MOER stabilizes reinforcement learning by injecting a decaying expert chain-of-thought prefix while allowing for model self-exploration. A selective rollback mechanism replaces failed completions with full expert traces to ensure trajectory correctness during training.
HOW THIS AFFECTS YOU
●
researcherYou can use this framework to improve RL training stability when incorporating off-policy expert data.