Mechanistic Analysis of Singular and Plural Coreference in LLMs

Mechanistic interpretability reveals specific attention heads in LLMs responsible for representing, identifying, and transferring plural entity information for pronoun prediction. Findings show LLMs utilize ontological similarity and linguistic links to determine when entities form a plural reference.