Interpreting Chain of Thought via Behavioral Counterfactuals

A new method localizes how prompt interventions affect reasoning by training a module to edit the residual stream. This approach minimizes KL divergence between output distributions of source and counterfactual prompts, outperforming standard activation reconstruction for recovering behavior.