Aligning Decoder-Only LLMs via MoE Router Outputs

To achieve cross-lingual alignment in decoder-only models despite varying tokenization, this method uses Mixture-of-Experts (MoE) router outputs as alignment targets. This approach enables more reliable cross-lingual co-occurrence modeling than applying auxiliary loss directly to hidden states.