DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism

DisagMoE overlaps computation and all-to-all communication in MoE training via disaggregated AF-Pipe parallelism to reduce network-bound stalls when distributing trillion-parameter expert models across GPU nodes.