Revealing Algorithmic Deductive Circuits for Logical Reasoning

Mechanistic interpretability work identifies specific attention heads responsible for individual deductive reasoning steps in LLMs, using symbolic-aided Chain-of-Thought prompting to align reasoning steps with token logits. The study maps information flow between attention heads during graph traversal tasks, revealing how models internally implement multi-step logical inference from few-shot demonstrations.