Aurora2: 400M sparse MoE model trained in 36 hours
November 18, 2026
Aurora2 is a 400M parameter sparse Mixture-of-Experts model with 32 experts, where 1 expert is active per token. It was pretrained on 100B tokens using a single RTX Pro 6000 in approximately 36 hours.
HOW THIS AFFECTS YOU
●
builderYou can train a functional MoE model on a single professional workstation in a few days.
●
researcherThis demonstrates high efficiency using Muon and AdamW optimizers on small-scale hardware.