Huawei has open-sourced OpenPangu-2.0-Flash, a Mixture-of-Experts model with 92B total parameters and 6B active parameters. The release includes weights, inference code, and training operations, supporting a 512K context window.
HOW THIS AFFECTS YOU
●
builderYou can deploy a high-context MoE model with relatively low active parameter requirements for efficient inference.
●
researcherThe release of training operations and weights allows for deeper study of large-scale MoE scaling.