Preventing Actor-Side Failure in No-Retention Offline-to-Online RL

This method improves fine-tuning performance in offline-to-online reinforcement learning when the offline dataset cannot be retained. By stopping online behavioral fitting while preserving the pretrained expressive prior, the authors mitigate distribution shifts across IFQL, FQL, and QC architectures on OGBench domains.