Joint Training for Dual Single-Pass and Looped Transformer Inference

Adapting pretrained models to support looped inference by reusing middle-layer weights can degrade single-pass performance. Joint training using weighted losses and guidance from the frozen original model allows models to improve looped predictions while maintaining single-pass quality within predefined tolerances.