Human-like fleeting memory improves language learning but impairs reading time prediction in transformer language models

Implementing fleeting memory in transformer models improves language modeling performance and syntactic evaluation on developmentally realistic datasets. While this memory constraint aids language acquisition, it simultaneously degrades the model's ability to predict surprisal-based reading times. These results suggest that architectural recency biases may facilitate learning at the expense of precise temporal prediction.