Data Diversity Drives Symbolic Mechanism Emergence in LLMs

Training experiments show that increased data diversity produces stronger symbolic signatures in transformer-based models. These mechanisms for abstract reasoning emerged even in multilayer perceptrons, suggesting architectural inductive biases are not strictly required.