Analyzing LLM syntax representation via linear distance and entropy

Research identifies that syntactic reconstruction accuracy in LLMs is predicted by the linear distance between words and the similarity-aware entropy of the syntactic relation's head. These findings disaggregate structural probe performance across different linguistic relations.