LIMe Extension Increases Transformer Representation Capacity

LIMe (Layer-Integrated Memory) is a lightweight extension that integrates representations from previous layers using learned, per-head, per-layer routing weights. This design addresses representation collapse and improves perplexity per FLOP in language modeling and synthetic reasoning tasks.