Measuring LLM Explanation Self-Consistency via Perturbation Strength

A new LLM-as-a-judge approach quantifies the strength of input and Chain-of-Thought perturbations to evaluate explanation self-consistency. Results show that input perturbations impact model consistency more significantly than CoT changes, suggesting current self-consistency metrics may be biased by perturbation type.