Taylor-Calibrate Improves Transformer-to-Linear-Attention Distillation Initialization

Taylor-Calibrate uses Taylor expansion to initialize Gated DeltaNet recurrent parameters (decay, write, output-gating) when converting pretrained Transformers to hybrid linear attention models, avoiding the poor dynamical regimes that cause distillation instability. The method reduces the token budget needed to recover teacher behavior compared to naive projection copying.