Token collapse identified in textual graph soft prompting
October 21, 2026
An analysis of G-Retriever reveals a token collapse phenomenon where graph-specific tokens become nearly indistinguishable across different inputs. This failure persists across multiple LLM families and both frozen and LoRA-adapted models.
HOW THIS AFFECTS YOU
●
researcherYou should be cautious when using graph soft prompting, as relational information may not be properly preserved.