Mechanisms of In-Context Learning Acquisition and Composition

A diagnostic study across 17 models reveals that in-context learning transitions from simple copying of demonstrations to rule-consistent composition. Checkpoint analyses show models prioritize task-relevant tokens long before accuracy improves beyond chance levels.