Position Bias Patterns Across Transformer and State-Space Architectures
October 23, 2026
Analysis of long-context architectures shows Transformers exhibit a beginning-of-sequence advantage, while both Transformers and state-space models favor evidence near the end of a sequence. Findings indicate that prompt changes can significantly alter or mitigate these positional biases across different model families.
HOW THIS AFFECTS YOU
●
researcherYou should account for architecture-specific position biases when designing long-context retrieval and RAG benchmarks.