vLLM has added support for EAGLE3/DFlash auxiliary hidden states specifically for GraniteMoe and GraniteMoeShared architectures. This improves inference efficiency for these models.
HOW THIS AFFECTS YOU
●
builderYou can achieve faster inference speeds when serving GraniteMoe models using vLLM.