Retraining-free elastic transformers adapt dynamically to varying computational budgets

A new method enables pre-trained transformers to scale across a continuous range of hardware constraints without retraining. By jointly ranking every individual MLP neuron and attention head on a common scale, the model can adjust its size on-the-fly to optimize for latency or throughput.