LLM agent credit assignment signals fail to outperform chance

Auditing step-level credit signals—including LLM-judge scores and logprob ratios—against causal ground truth in ALFWorld shows these methods fail to identify causally significant steps better than chance. Causal contribution is found to be sparse, affecting only 30.5% of decision points.