How Do Language Models Represent and Use Phonological Information for Allomorph Selection?

Language models encode phonological conditions for English article selection along a single linear direction within trigger-token embeddings. This direction causally drives allomorph selection by utilizing forecasted phonological features of upcoming tokens during prediction. These findings suggest models move beyond simple memorization to implement rule-like generalization for phonologically conditioned morphemes.