A new update to vLLM introduces masked_token_ids, allowing users to specify tokens that should never be sampled during inference. This feature provides more granular control over model output generation.
HOW THIS AFFECTS YOU
●
builderYou can prevent your models from generating specific unwanted tokens.