Source Observation During Pretraining Improves Reward Adaptation in LLMs

This study demonstrates that providing task-independent source observations during pretraining allows models to better utilize rewards for adaptation. Experiments with Qwen2.5 show that models with source supervision achieve 82.61% success in sequential tasks compared to 44.15% without it.