Untrained MTP Head Causes Latency in Ornith 1.5 35B
August 20, 2026
Ornith-1.5-35B-A3B is currently shipping with a Multi-Token Prediction (MTP) head that has random initialization rather than training. This configuration explains the significant performance slowdown observed in the model.
HOW THIS AFFECTS YOU
●
builderAvoid deploying this specific model version for latency-sensitive applications until the MTP head is properly trained.
●
researcherVerify MTP head weight initialization in open-source distributions before benchmarking architectural speedups.