Identification of Interpretable Narrative Mechanisms in LLMs

Analysis of GPT-2, Gemma-2, and Llama-3.1 reveals specific attention heads and MLP neurons that represent narrativity. Ablating these features reduces the narrative quality of model generations and increases prediction loss specifically on high-narrativity text.