Using Hidden States Instead of Semantic Embeddings Improves LLM Routing

Research finds that semantic embeddings are unreliable proxies for LLM routing on unseen tasks due to geometric failure modes. Replacing semantic embeddings with the last-token hidden state from a partial Qwen2.5-7B-Instruct forward pass improves routing generalization and cost-adjusted performance.