Improving RL Agent Exploration via Observation Supervision
September 16, 2026
The ActObs method supervises observation tokens in agent trajectories during SFT, rather than just action tokens. On Qwen3-4B, this approach leads to higher pass@k rates during GRPO by encouraging the policy to model action consequences without increasing parameter counts.
HOW THIS AFFECTS YOU
●
builderYou can improve agent performance during RL fine-tuning without increasing model size or compute overhead.
●
researcherSupervising observation tokens provides a more effective initialization for reinforcement learning.