KAPRO Framework Exposes LLM Agent Self-Awareness Gaps in Tool Use

KAPRO benchmarks LLM agents on metacognitive alignment — whether they correctly judge when to use external tools versus internal knowledge — using the KAware dataset partitioned into external, internal, and hybrid task subspaces. Results show self-awareness correlates strongly with task success but degrades significantly in internal-knowledge settings, with open-source models underperforming.