Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training

Training a single transformer layer can recover most performance gains from full-parameter reinforcement learning (RL) post-training. Measuring layer contribution shows that RL adaptation is highly localized, allowing for efficient single-layer updates that occasionally surpass the results of updating all model parameters.