Ordinal Addressing Hypothesis: LLMs Locate Facts by Order of Mention
October 2, 2026
Research reveals that LLMs represent facts in context via an ordinal vector, pointing to a fact's order of mention rather than its specific identifier. This ordinal addressing hypothesis suggests that hidden states move to a specific fact address based on the sequence in which information was provided.
HOW THIS AFFECTS YOU
●
researcherUnderstanding how order affects internal state representations is critical for improving in-context learning and retrieval accuracy.