A new update to vLLM adds validation for the runai_streamer model_loader_extra_config field. This prevents misconfigurations in streaming setups when using Run:ai.
HOW THIS AFFECTS YOU
●
builderConfiguration errors in your Run:ai streaming pipelines will be caught during the loading phase.