The Baton family of looped transformers improves two-hop reasoning by passing intermediate entity embeddings to a second loop. While standard transformers struggle with these tasks, Baton variants can answer them almost perfectly when trained on the answer alone.
HOW THIS AFFECTS YOU
●
researcherYou can use looped architectures to pass intermediate information through token embeddings.