Analysis of four open-weight typed decision models reveals a preference for label strings over provided definitions, a phenomenon termed option-label bias. This occurs even when using Qwen2.5 backbones for classification tasks.
HOW THIS AFFECTS YOU
●
builderBe cautious when using LLMs for routing; ensure your label names don't inadvertently bias the model's logic.
●
researcherThis identifies a specific failure mode in instruction-following for classification.