Nemotron 3 Ultra: 550B MoE Hybrid Mamba-Transformer With 6x Inference Throughput Gain

Nemotron 3 Ultra is a 550B total / 55B active parameter MoE model combining Mamba and attention layers, pretrained on 20T tokens with 1M token context, achieving up to 6x higher inference throughput than comparable open LLMs at equivalent accuracy. It uses LatentMoE, NVFP4 pretraining, multi-teacher on-policy distillation, and reasoning budget control, and is fully open.