Muon Optimizer Increases MoE Layer Balancing Difficulty

Experiments on MoE models with 55M and 108M active parameters show that the Muon optimizer roughly doubles the span of routing biases in the first MoE layer. This indicates that Muon makes the load balancing of the first layer significantly harder.