Diagnosing Tool-Selection Reasoning in LLM Agents with Canary Tools

Canary tools enable diagnostic profiling of LLM agent tool-selection errors by embedding probes into Model Context Protocol sets. Using a six-type taxonomy including semantic decoys and parameter traps, the method converts binary failures into multidimensional reasoning profiles. Evaluations across eight models show susceptibility to these errors decreases sharply with model scale.