Multiscale Policy Optimization (MPO) enhances long-horizon agent performance by aligning credit assignment with turn-level importance sampling. Experiments on 4B and 30B-A3B models show consistent gains in search, tool-use, and automation tasks over existing RL baselines.
HOW THIS AFFECTS YOU
●
builderThis offers a more effective RL approach for training agents that must complete long, multi-step tasks.
●
researcherThe method provides a way to resolve the discrepancy between token-level and turn-level critic values.