Natural-Language Training vs. Code Training for Brain-Alignment

Training GPT-2-small models on TinyStories results in higher brain scores on Pereira-243 and Tuckute2024 compared to models trained on Python, C, assembly, or machine code. The research shows that code training does not substitute for natural-language training in achieving brain-aligned geometry.