Effective Learning Rate Scheduling via Affine Inverse-square-Root Decay
October 28, 2026
Weight-normalized training gains are driven by effective learning rate (ELR) trajectories rather than normalization alone. The proposed AIR schedule matches ELR dynamics by combining 1/t decay with an affine adjustment to satisfy peak and terminal constraints.
HOW THIS AFFECTS YOU
●
researcherYou should focus on designing schedules based on ELR trajectories rather than nominal learning rates when using weight normalization.