Mechanistic Interpretation of Reasoning Operations in LLM Representations

Research into the geometric organization of LLM reasoning shows that operations like goal decomposition and deduction are separable in hidden representations. This separability peaks in the middle layers and is distinct from lexical or positional patterns.