Periodic Row-wise Muon Optimizes Large Diffusion Transformers

Scaling the Muon optimizer to 15B parameter Diffusion Transformers reveals high computation overhead from Newton-Schulz iterations. Periodic Row-wise Muon mitigates this by performing full spectral updates every K steps and applying low-cost row-wise constrained updates in between, preserving optimization advantages while reducing communication costs.