LLM Abstention Fails to Follow Prompted Probability Thresholds
November 13, 2026
Probing 3-32B parameter models shows that explicit threshold instructions in prompts fail to regulate abstention rates effectively. While an abstention format increased mean abstention from 20% to 91% at a 0.75 threshold, raising the threshold from 0.5 to 0.9 changed the rate by only 0.8 percentage points.
HOW THIS AFFECTS YOU
●
builderDo not trust a model to accurately self-select when to say 'I don't know' using only a prompt.
●
researcherYou cannot rely on simple prompting to force models to abstain based on specific confidence thresholds.