Transformer Architectures Exhibit Linear Superposition of Input Streams

Transformers exhibit a Superposition Linearity Hypothesis where combining distinct text streams results in a superposition of next-token distributions. This intrinsic architectural property diminishes during pretraining but can be restored via lightweight fine-tuning to minimize distribution divergence.