SCQ Stabilizes Offline-to-Online Reinforcement Learning via Sigmoid-Bounded Entropy
September 14, 2026
SCQ replaces standard log-entropy with a sigmoid-bounded formulation to prevent negative values from destabilizing policy updates in conservative Q-learning. The method maintains conservative regularization and achieves performance parity or better on D4RL and real-world visual benchmarks.
HOW THIS AFFECTS YOU
●
researcherYou can use this bounded entropy formulation to solve value estimation instability in offline-to-online RL.