GraphACM: Graph-based Action Consistency Measurement for Uncertainty Quantification in LLM Agents

GraphACM quantifies LLM agent uncertainty by merging task rollouts into directed, action-labeled graphs to compare decisions at equivalent states. This framework moves beyond single-score summaries to provide node-level consistency measures. Testing on ALFWorld, AppWorld, and WebShop shows global consistency scores reliably predict task success or failure.