Transformers Are Inherently Succinct

Transformer architectures are provably parameter-efficient by construction: the paper establishes theoretical bounds showing transformers require asymptotically fewer parameters than alternative architectures to represent the same function class. The work earned one of three outstanding paper awards at ICLR 2026.