Semantic Entropy Improves LLM Routing Accuracy via Uncertainty Signals
October 7, 2026
Semantic entropy reliably identifies small model failures on GSM8K with an AUROC of 0.871, improving routed accuracy by up to nine points at matched cost. The study warns against using synthetic benchmarks that rely solely on question difficulty rather than model-specific uncertainty.
HOW THIS AFFECTS YOU
●
builderYou can use semantic entropy to decide when to escalate queries from small to large models to optimize cost and accuracy.
●
researcherBe cautious of routing benchmarks that do not account for question difficulty as a confounding variable.