Cross-Lingual Alignment via MoE Router Outputs

Aligning decoder-only models across languages is hindered by varying tokenization. This method uses Mixture-of-Experts (MoE) router outputs as a sequence-level alignment target instead of applying auxiliary loss to hidden states, improving cross-lingual transfer during continual pre-training.