vLLM v0.28.0 introduces Kimi-K3 optimizations, including Decode Context Parallel support and fused kernels. Key updates include a 60% improvement in DSpark TTFT via adaptive speculative token budgeting and memory savings of 17 GiB per GPU through shared-expert sharding.
HOW THIS AFFECTS YOU
●
builderYou can achieve significantly lower TTFT and higher memory efficiency for large-scale deployments.
●
researcherThe new kernel-level speedups and sequence parallelism support enable more efficient large-scale model training and inference.