Gated Recurrent Transformers for Efficient Model Scaling

Gated Recurrent Transformers reduce memory footprints by replacing depth-heavy architectures with a single shared core iterated R times. The design uses a lightweight projection and an elementwise update gate conditioned on hidden states and noise to maintain representational diversity.