The Impossibility Triangle of Long-Context Modeling

This theoretical framework proves a fundamental trade-off between efficiency, compactness, and recall in long-sequence models. It demonstrates that no architecture can simultaneously maintain per-step computation, constant state size, and recall proportional to sequence length.