Reversal Curse: LLMs Fail to Invert Factual Associations from Training Data

LLMs trained on directional facts like 'A is B' fail to generalize to 'B is A', with the probability of the correct reversed answer no higher than a random baseline. The effect holds across model sizes and is not fixed by scale, pointing to a fundamental limitation in how autoregressive models encode relational knowledge.